You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by Vincenzo De Leo <na...@gmail.com> on 2015/09/30 15:17:29 UTC
hama streaming problem with hello world example
Hi to all,
this is my first mail to this mailing list so please have patience if I
make some bad choice in the format.
I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500 with 2 GB
of RAM.
I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e Hama-0.7.0
as you can see from the following lines:
>
> > $ hadoop version
> > Hadoop 2.7.1
> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
15ecc87ccf4a0228f35af08fc56de536e6ce657a
> > Compiled by jenkins on 2015-06-29T06:04Z
> > Compiled with protoc 2.5.0
> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
> > This command was run using
/home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
> > $ java -version
> > java version "1.8.0_60"
> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> >
I am able to properly run the basic Hadoop and Hama examples like the
following:
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
grep input output 'dfs[a-z.]+'
bin/hama jar hama-examples-0.7.0.jar pi
My problem is that I receive the following error message when I try to run
the HelloWorld Hama Streaming example with this instruction:
bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
-cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
/tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
The default python interpreters of my OS were python-2.7 and python-3.4;
since I had problems with this example I also tried to install python-3.2
with the following instructions but it didn't solved the problem:
>
> > sudo apt-get install software-properties-common
> > sudo apt-add-repository ppa:fkrull/deadsnakes
> > sudo apt-get update
> > sudo apt-get install python3.2
> >
The installed python version is 3.2.6 as you can see from the following
lines:
>
> > $ python3.2
> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
> > [GCC 4.8.2] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>>
> >
The error message is the following (I am working in local mode so I didn't
run bin/start-bspd.sh):
>
>
> >
> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
python3.2 -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
-program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
> > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name is
deprecated. Instead, use fs.defaultFS
> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
deprecated. Instead, use mapreduce.job.user.name
> > 15/09/30 12:39:11 WARN conf.Configuration:
org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
> > 15/09/30 12:39:11 WARN conf.Configuration:
org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
deprecated. Instead, use mapreduce.job.user.name
> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
job_localrunner_0001
> > 15/09/30 12:39:12 INFO Configuration.deprecation:
mapred.cache.localFiles is deprecated. Instead, use
mapreduce.job.cache.local.files
> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new barrier for
2 tasks!
> > java.lang.NumberFormatException: For input string: "Traceback (most
recent call last):"
> > at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > at java.lang.Integer.parseInt(Integer.java:580)
> > at java.lang.Integer.parseInt(Integer.java:615)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > java.lang.NumberFormatException: For input string: "Traceback (most
recent call last):"
> > at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > at java.lang.Integer.parseInt(Integer.java:580)
> > at java.lang.Integer.parseInt(Integer.java:615)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception: Bad
command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >
> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception: Bad
command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >
> > java.util.concurrent.BrokenBarrierException
> > at
java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > at
java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > at
org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
java.lang.Exception: Bad command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > Caused by: java.lang.Exception: Bad command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > java.util.concurrent.BrokenBarrierException
> > at
java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > at
java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > at
org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
java.lang.Exception: Bad command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > Caused by: java.lang.Exception: Bad command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
execution!
> > java.io.IOException: Stream closed
> > at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > at java.io.OutputStream.write(OutputStream.java:116)
> > at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
execution!
> > java.lang.InterruptedException: java.lang.Exception: Bad command code:
-2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >
> > at
org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> > at
org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
execution.
> > java.io.IOException: Stream closed
> > at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > at java.io.OutputStream.write(OutputStream.java:116)
> > at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
execution.
> > java.io.IOException: Stream closed
> > at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > at java.io.OutputStream.write(OutputStream.java:116)
> > at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > at
org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
execution!
> > java.util.concurrent.ExecutionException:
java.lang.InterruptedException: java.lang.Exception: Bad command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >
> > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> > at
org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.InterruptedException: java.lang.Exception: Bad
command code: -2
> > at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >
> > at
org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> > at
org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > ... 1 more
> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
> >
I tried to searching for these error messages with google but I didn't find
any solution.
Does anyone of you have any idea of how can I solve this problem?
Any help is appreciated :)
Thank you
Best
Vincenzo
Re: hama streaming problem with hello world example
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hi,
We are investigating the problem. By the way, I really don't know
where the string "Traceback (most recent call last):" comes from. When
I tested it, I always seeing -1.
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > at java.lang.Integer.parseInt(Integer.java:580)
> > at java.lang.Integer.parseInt(Integer.java:615)
> > at
On Wed, Oct 7, 2015 at 7:41 PM, Vincenzo De Leo <na...@gmail.com> wrote:
> Thank You Edward,
> can you explain to me how can I use your solution? I have tried with:
>
> - bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -inputformat
> org.apache.hama.bsp.NullInputFormat -outputformat
> org.apache.hama.bsp.SequenceFileOutputFormat -program
> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>
> but I received the same old error messages... Do I have to install some
> patch and how can I eventually do this?
>
> Vincenzo
>
> On Mon, Oct 5, 2015 at 2:41 AM, Edward J. Yoon <ed...@apache.org>
> wrote:
>
>> Hi Vincenzo,
>>
>> I just found some code changes related to this problem at
>> https://issues.apache.org/jira/browse/HAMA-839, and I was able to run
>> Python program by adding the in/output formatters like: -inputformat
>> org.apache.hama.bsp.NullInputFormat -outputformat
>> org.apache.hama.bsp.SequenceFileOutputFormat.
>>
>> I'll check more and update the Wiki page. Sorry for inconvenient!
>>
>>
>> On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>> > I can look at it closely next week. BTW, should input be provided?
>> >
>> > On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
>> > <vi...@linkalab.it> wrote:
>> >> Thank you for your kind reply; this is the output of the K-Means
>> example:
>> >>
>> >> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>> python3.2
>> >> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>> >> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
>> >> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop
>> >> library for your platform... using builtin-java classes where applicable
>> >> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
>> >> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
>> >> deprecated. Instead, use fs.defaultFS
>> >> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is
>> deprecated.
>> >> Instead, use mapreduce.job.user.name
>> >> 15/10/01 15:06:57 WARN conf.Configuration:
>> >>
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> :an
>> >> attempt to override final parameter:
>> >> mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> >> 15/10/01 15:06:57 WARN conf.Configuration:
>> >>
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> :an
>> >> attempt to override final parameter:
>> >> mapreduce.job.end-notification.max.attempts; Ignoring.
>> >> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is
>> deprecated.
>> >> Instead, use mapreduce.job.user.name
>> >> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job:
>> job_localrunner_0001
>> >> 15/10/01 15:06:57 INFO Configuration.deprecation:
>> mapred.cache.localFiles
>> >> is deprecated. Instead, use mapreduce.job.cache.local.files
>> >> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier for
>> 2
>> >> tasks!
>> >> java.lang.NumberFormatException: For input string: "Traceback (most
>> recent
>> >> call last):"
>> >> at
>> >>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >> at java.lang.Integer.parseInt(Integer.java:580)
>> >> at java.lang.Integer.parseInt(Integer.java:615)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >> java.lang.NumberFormatException: For input string: "Traceback (most
>> recent
>> >> call last):"
>> >> at
>> >>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >> at java.lang.Integer.parseInt(Integer.java:580)
>> >> at java.lang.Integer.parseInt(Integer.java:615)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
>> >> command code: -2
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>
>> >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> >> java.lang.Exception: Bad command code: -2
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >> Caused by: java.lang.Exception: Bad command code: -2
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> java.util.concurrent.BrokenBarrierException
>> >> at
>> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >> at
>> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >> at
>> >> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
>> >> command code: -2
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>
>> >> java.util.concurrent.BrokenBarrierException
>> >> at
>> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >> at
>> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >> at
>> >> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> >> java.lang.Exception: Bad command code: -2
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >> Caused by: java.lang.Exception: Bad command code: -2
>> >> at
>> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> execution!
>> >> java.io.IOException: Stream closed
>> >> at
>> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> at java.io.OutputStream.write(OutputStream.java:116)
>> >> at
>> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> execution!
>> >> java.io.IOException: Stream closed
>> >> at
>> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> at java.io.OutputStream.write(OutputStream.java:116)
>> >> at
>> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
>> >> execution.
>> >> java.io.IOException: Stream closed
>> >> at
>> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> at java.io.OutputStream.write(OutputStream.java:116)
>> >> at
>> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> execution!
>> >> java.util.concurrent.ExecutionException: java.io.IOException: Stream
>> closed
>> >> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> >> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> Caused by: java.io.IOException: Stream closed
>> >> at
>> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> at java.io.OutputStream.write(OutputStream.java:116)
>> >> at
>> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> ... 1 more
>> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
>> >> execution.
>> >> java.io.IOException: Stream closed
>> >> at
>> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> at java.io.OutputStream.write(OutputStream.java:116)
>> >> at
>> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> at
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> at
>> >>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> at
>> >>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
>> >>
>> >> Vincenzo
>> >>
>> >> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <ed...@apache.org>
>> >> wrote:
>> >>
>> >>> It looks like a bug of input reader. Can you try to run K-Means
>> example?
>> >>>
>> >>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
>> >>> <na...@gmail.com> wrote:
>> >>> > Hi to all,
>> >>> > this is my first mail to this mailing list so please have patience
>> if I
>> >>> > make some bad choice in the format.
>> >>> >
>> >>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500
>> with 2
>> >>> GB
>> >>> > of RAM.
>> >>> >
>> >>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
>> >>> Hama-0.7.0
>> >>> > as you can see from the following lines:
>> >>> >
>> >>> >
>> >>> >>
>> >>> >> > $ hadoop version
>> >>> >> > Hadoop 2.7.1
>> >>> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
>> >>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
>> >>> >> > Compiled by jenkins on 2015-06-29T06:04Z
>> >>> >> > Compiled with protoc 2.5.0
>> >>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
>> >>> >> > This command was run using
>> >>> >
>> >>>
>> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
>> >>> >
>> >>> >
>> >>> >> > $ java -version
>> >>> >> > java version "1.8.0_60"
>> >>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
>> >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>> >>> >> >
>> >>> >
>> >>> > I am able to properly run the basic Hadoop and Hama examples like the
>> >>> > following:
>> >>> > bin/hadoop jar
>> share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
>> >>> > grep input output 'dfs[a-z.]+'
>> >>> > bin/hama jar hama-examples-0.7.0.jar pi
>> >>> > My problem is that I receive the following error message when I try
>> to
>> >>> run
>> >>> > the HelloWorld Hama Streaming example with this instruction:
>> >>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>> >>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>> >>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> >>> > The default python interpreters of my OS were python-2.7 and
>> python-3.4;
>> >>> > since I had problems with this example I also tried to install
>> python-3.2
>> >>> > with the following instructions but it didn't solved the problem:
>> >>> >
>> >>> >
>> >>> >>
>> >>> >> > sudo apt-get install software-properties-common
>> >>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
>> >>> >> > sudo apt-get update
>> >>> >> > sudo apt-get install python3.2
>> >>> >> >
>> >>> >
>> >>> >
>> >>> > The installed python version is 3.2.6 as you can see from the
>> following
>> >>> > lines:
>> >>> >
>> >>> >
>> >>> >>
>> >>> >> > $ python3.2
>> >>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
>> >>> >> > [GCC 4.8.2] on linux2
>> >>> >> > Type "help", "copyright", "credits" or "license" for more
>> information.
>> >>> >> > >>>
>> >>> >> >
>> >>> >
>> >>> >
>> >>> > The error message is the following (I am working in local mode so I
>> >>> didn't
>> >>> > run bin/start-bspd.sh):
>> >>> >
>> >>> >
>> >>> >>
>> >>> >>
>> >>> >> >
>> >>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>> >>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output
>> /tmp/pystream-out/
>> >>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> >>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
>> >>> > native-hadoop library for your platform... using builtin-java classes
>> >>> where
>> >>> > applicable
>> >>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
>> >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name
>> is
>> >>> > deprecated. Instead, use fs.defaultFS
>> >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
>> >>> > deprecated. Instead, use mapreduce.job.user.name
>> >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> >>> >
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> >>> :an
>> >>> > attempt to override final parameter:
>> >>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> >>> >
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> >>> :an
>> >>> > attempt to override final parameter:
>> >>> > mapreduce.job.end-notification.max.attempts; Ignoring.
>> >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
>> >>> > deprecated. Instead, use mapreduce.job.user.name
>> >>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
>> >>> > job_localrunner_0001
>> >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
>> >>> > mapred.cache.localFiles is deprecated. Instead, use
>> >>> > mapreduce.job.cache.local.files
>> >>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new
>> barrier
>> >>> for
>> >>> > 2 tasks!
>> >>> >> > java.lang.NumberFormatException: For input string: "Traceback
>> (most
>> >>> > recent call last):"
>> >>> >> > at
>> >>> >
>> >>>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >>> >> > java.lang.NumberFormatException: For input string: "Traceback
>> (most
>> >>> > recent call last):"
>> >>> >> > at
>> >>> >
>> >>>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>> java.lang.Exception:
>> >>> Bad
>> >>> > command code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> >> >
>> >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>> java.lang.Exception:
>> >>> Bad
>> >>> > command code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> >> >
>> >>> >> > java.util.concurrent.BrokenBarrierException
>> >>> >> > at
>> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >>> >> > at
>> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >>> >> > at
>> >>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> >> > Exception in thread "pipe-uplink-handler"
>> java.lang.RuntimeException:
>> >>> > java.lang.Exception: Bad command code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> >> > java.util.concurrent.BrokenBarrierException
>> >>> >> > at
>> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >>> >> > at
>> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >>> >> > at
>> >>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> >> > Exception in thread "pipe-uplink-handler"
>> java.lang.RuntimeException:
>> >>> > java.lang.Exception: Bad command code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>> >>> > execution!
>> >>> >> > java.io.IOException: Stream closed
>> >>> >> > at
>> >>> >
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >>> >> > at
>> >>> >
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> >> > at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> >> > at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> >> > at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >>> >> > at
>> >>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>> >>> > execution!
>> >>> >> > java.lang.InterruptedException: java.lang.Exception: Bad command
>> code:
>> >>> > -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> >> >
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> >>> >> > at
>> >>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
>> after
>> >>> bsp
>> >>> > execution.
>> >>> >> > java.io.IOException: Stream closed
>> >>> >> > at
>> >>> >
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >>> >> > at
>> >>> >
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> >> > at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> >> > at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> >> > at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >>> >> > at
>> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >>> >> > at
>> >>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
>> after
>> >>> bsp
>> >>> > execution.
>> >>> >> > java.io.IOException: Stream closed
>> >>> >> > at
>> >>> >
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >>> >> > at
>> >>> >
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> >> > at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> >> > at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> >> > at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >>> >> > at
>> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >>> >> > at
>> >>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>> >>> > execution!
>> >>> >> > java.util.concurrent.ExecutionException:
>> >>> > java.lang.InterruptedException: java.lang.Exception: Bad command
>> code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> >> >
>> >>> >> > at
>> java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> >>> >> > at
>> java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> >> > Caused by: java.lang.InterruptedException: java.lang.Exception:
>> Bad
>> >>> > command code: -2
>> >>> >> > at
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> >> >
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> >>> >> > at
>> >>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> >> > at
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> >> > at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> >> > at
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> >> > ... 1 more
>> >>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
>> >>> >> >
>> >>> >
>> >>> >
>> >>> > I tried to searching for these error messages with google but I
>> didn't
>> >>> find
>> >>> > any solution.
>> >>> >
>> >>> > Does anyone of you have any idea of how can I solve this problem?
>> >>> >
>> >>> > Any help is appreciated :)
>> >>> >
>> >>> > Thank you
>> >>> > Best
>> >>> >
>> >>> > Vincenzo
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best Regards, Edward J. Yoon
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >>
>> -----------------------------------------------------------------------------
>> >> Dr. Vincenzo De Leo, PhD
>> >> Research and Development @ Linkalab
>> >>
>> -----------------------------------------------------------------------------
>> >> Home Page: http://www.linkalab.it/people/vincenzo.deleo
>> >> e-mail: vincenzo.deleo@linkalab.it
>> >> Phone: (+39)0706753140
>> >> Mobile: (+39)3406673544
>> >> Skype contact: vincenzo.deleo77
>> >> GTalk contact: nagatobimaru
>> >>
>> -----------------------------------------------------------------------------
>> >> Linkalab (http://www.linkalab.it)
>> >> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
>> >> Via San Benedetto, 88
>> >> 09129 Cagliari (CA), ITALY
>> >> P. IVA: 03148020922
>> >> Phone: (+39) 0707966021
>> >> FAX: (+39) 0707962000
>> >>
>> -----------------------------------------------------------------------------
>> >
>> >
>> >
>> > --
>> > Best Regards, Edward J. Yoon
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>>
--
Best Regards, Edward J. Yoon
Re: hama streaming problem with hello world example
Posted by "Edward J. Yoon" <ed...@apache.org>.
I've fixed minor bug, Please use the latest trunk. :-)
Here's my full log for you:
% git clone git://github.com/thomasjungblut/HamaStreaming.git
% cd hadoop-2.7.1
% bin/hadoop dfs -mkdir /tmp/PyStreaming
% bin/hadoop dfs -copyFromLocal ../HamaStreaming/* /tmp/PyStreaming/
% git clone https://git-wip-us.apache.org/repos/asf/hama.git
% mvn clean install -DskipTests=true
% cp ../hama-stable/conf/** ./conf/
% bin/start-bspd.sh
cluster-0: starting zookeeper, logging to
/root/hama/bin/../logs/hama-root-zookeeper-cluster-0.out
starting bspmaster, logging to
/root/hama/bin/../logs/hama-root-bspmaster-cluster-0.out
cluster-0: starting groom, logging to
/root/hama/bin/../logs/hama-root-groom-cluster-0.out
% bin/hama jar examples/target/hama-examples-0.7.1-SNAPSHOT.jar pi
15/11/24 15:57:19 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable
15/11/24 15:57:20 INFO Configuration.deprecation: user.name is
deprecated. Instead, use mapreduce.job.user.name
15/11/24 15:57:20 INFO bsp.BSPJobClient: Running job: job_201511241557_0001
15/11/24 15:57:23 INFO bsp.BSPJobClient: Current supersteps number: 0
15/11/24 15:57:26 INFO bsp.BSPJobClient: Current supersteps number: 1
15/11/24 15:57:26 INFO bsp.BSPJobClient: The total number of supersteps: 1
15/11/24 15:57:26 INFO bsp.BSPJobClient: Counters: 6
15/11/24 15:57:26 INFO bsp.BSPJobClient:
org.apache.hama.bsp.JobInProgress$JobCounter
15/11/24 15:57:26 INFO bsp.BSPJobClient: SUPERSTEPS=1
15/11/24 15:57:26 INFO bsp.BSPJobClient: LAUNCHED_TASKS=3
15/11/24 15:57:26 INFO bsp.BSPJobClient:
org.apache.hama.bsp.BSPPeerImpl$PeerCounter
15/11/24 15:57:26 INFO bsp.BSPJobClient: SUPERSTEP_SUM=3
15/11/24 15:57:26 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=86
15/11/24 15:57:26 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=3
15/11/24 15:57:26 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=3
Estimated value of PI is 3.1472
Job Finished in 6.816 seconds
% python --version
Python 2.7.5
% bin/hama pipes -streaming true -bspTasks 2 -interpreter python
-cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-edward/
-program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
15/11/24 15:57:34 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable
15/11/24 15:57:35 INFO pipes.Submitter: Streaming enabled!
15/11/24 15:57:35 INFO Configuration.deprecation: fs.default.name is
deprecated. Instead, use fs.defaultFS
15/11/24 15:57:35 WARN bsp.BSPJobClient: No job jar file set. User
classes may not be found. See BSPJob#setJar(String) or check Your jar
file.
15/11/24 15:57:35 INFO Configuration.deprecation: user.name is
deprecated. Instead, use mapreduce.job.user.name
15/11/24 15:57:35 INFO bsp.BSPJobClient: Running job: job_201511241557_0002
15/11/24 15:57:39 INFO bsp.BSPJobClient: Job complete: job_201511241557_0002
15/11/24 15:57:39 INFO bsp.BSPJobClient: The total number of supersteps: 15
15/11/24 15:57:39 INFO bsp.BSPJobClient: Counters: 6
15/11/24 15:57:39 INFO bsp.BSPJobClient:
org.apache.hama.bsp.JobInProgress$JobCounter
15/11/24 15:57:39 INFO bsp.BSPJobClient: LAUNCHED_TASKS=2
15/11/24 15:57:39 INFO bsp.BSPJobClient:
org.apache.hama.bsp.BSPPeerImpl$PeerCounter
15/11/24 15:57:39 INFO bsp.BSPJobClient: SUPERSTEP_SUM=30
15/11/24 15:57:39 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=507
15/11/24 15:57:39 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=60
15/11/24 15:57:39 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=60
15/11/24 15:57:39 INFO bsp.BSPJobClient: TASK_OUTPUT_RECORDS=56
Then you'll see like this.
Thanks.
On Tue, Nov 24, 2015 at 8:47 AM, Vincenzo De Leo
<vi...@linkalab.it> wrote:
> Hello Edward,
> I'm sorry but in this month I had to work at other stuff so I had the time
> to come back to try solve this problem only now.
>
> I followed your hints and started working in the Pseudo Distributed Mode
> but I still receive some new error messages.
>
> With respect to the original information that I gave you I modified the Sun
> Java JDK version (1.7.0_55 and not 1.8) python version (3.2.3 and not 3.4),
> hama version (now I am using hama-trunk and no more hama-0.7.0) and
> installed the hama-patch 805 (https://issues.apache.org/jira/browse/HAMA-805)
> also if this last return to me some error message so I am not exactly sure
> to have installed properly; these error messages are described below:
>
> I copied HAMA-805.patch in the same directory of hama-trunk then run
> patch -p1 < HAMA-805.patch
> and received these messages:
> hadoop_admin@tora-VirtualBox:~/hama-trunk$ patch -p1 < HAMA-805.patch
> can't find file to patch at input line 5
> Perhaps you used the wrong -p or --strip option?
> The text leading up to this was:
> --------------------------
> |Index:
> core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
> |===================================================================
> |---
> core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
> (revision 1526180)
> |+++
> core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
> (working copy)
> --------------------------
> File to patch:
> /home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
> patching file
> /home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
> Hunk #1 FAILED at 35.
> Hunk #2 succeeded at 66 with fuzz 2 (offset 5 lines).
> Hunk #3 FAILED at 272.
> 2 out of 3 hunks FAILED -- saving rejects to file
> /home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java.rej
> can't find file to patch at input line 45
> Perhaps you used the wrong -p or --strip option?
> The text leading up to this was:
> --------------------------
> |Index:
> core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
> |===================================================================
> |---
> core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
> (revision 1526180)
> |+++
> core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
> (working copy)
> --------------------------
> File to patch:
> /home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
> patching file
> /home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
> Hunk #1 succeeded at 40 with fuzz 1 (offset 1 line).
> Hunk #2 FAILED at 78.
> Hunk #3 FAILED at 117.
> 2 out of 3 hunks FAILED -- saving rejects to file
> /home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java.rej
>
> In any case I restarted the bspmaster and groomserver and submit again the
> same job:
>
> - hadoop_admin@tora-VirtualBox:~/hama-trunk$ bin/hama pipes -streaming
> true -bspTasks 2 -interpreter python3.2 -cachefiles /tmp/PyStreaming/*.py
> -output /tmp/pystream-out/ -program /tmp/PyStreaming/BSPRunner.py
> -programArgs HelloWorldBSP
>
> This time it seems that hama have properly read-access to /tmp/PyStreaming
> but it seems that it has no write-access to /tmp/pystream-out/:
>
> 15/11/24 00:01:42 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 15/11/24 00:01:44 INFO ipc.Server: Starting Socket Reader #1 for port 61002
> 15/11/24 00:01:44 INFO ipc.Server: IPC Server Responder: starting
> 15/11/24 00:01:44 INFO ipc.Server: IPC Server listener on 61002: starting
> 15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 1 on 61002: starting
> 15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 2 on 61002: starting
> 15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 3 on 61002: starting
> 15/11/24 00:01:44 INFO message.HamaMessageManagerImpl: BSPPeer
> address:localhost port:61002
> 15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 4 on 61002: starting
> 15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 0 on 61002: starting
> 15/11/24 00:01:44 FATAL bsp.GroomServer: Error running child
> java.io.FileNotFoundException: /tmp/pystream-out/part-00001 (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:212)
> at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:222)
> at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:209)
> at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:305)
> at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:294)
> at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:326)
> at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.(ChecksumFileSystem.java:393)
> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:776)
> at org.apache.hama.bsp.TextOutputFormat.getRecordWriter(TextOutputFormat.java:112)
> at org.apache.hama.bsp.BSPPeerImpl.initializeIO(BSPPeerImpl.java:329)
> at org.apache.hama.bsp.BSPPeerImpl.(BSPPeerImpl.java:184)
> at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1251)
> java.io.FileNotFoundException: /tmp/pystream-out/part-00001 (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:212)
> at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:222)
> at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:209)
> at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:305)
> at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:294)
> at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:326)
> at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.(ChecksumFileSystem.java:393)
> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
> at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:776)
> at org.apache.hama.bsp.TextOutputFormat.getRecordWriter(TextOutputFormat.java:112)
> at org.apache.hama.bsp.BSPPeerImpl.initializeIO(BSPPeerImpl.java:329)
> at org.apache.hama.bsp.BSPPeerImpl.(BSPPeerImpl.java:184)
> at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1251)
>
> I also tryied to set full permissions to /tmp/pystream-out with hadoop fs
> -chmod 777 /tmp/pystream-out:
>
> hadoop_admin@tora-VirtualBox:~/hama-trunk$ hadoop fs -ls /tmp
> 15/11/24 00:13:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Found 3 items
> drwxr-xr-x - hadoop_admin supergroup 0 2015-11-23 22:38
> /tmp/PyStreaming
> drwxr-xr-x - hadoop_admin supergroup 0 2015-11-23 22:40
> /tmp/hadoop-tora
> drwxrwxrwx - hadoop_admin supergroup 0 2015-11-23 23:17
> /tmp/pystream-out
>
> but the error message did not changed.
>
> Any help is appreciated,
> thank you very much,
> bye!
>
> Vincenzo
>
>
> On Thu, Oct 8, 2015 at 3:15 AM, Edward J. Yoon <ed...@apache.org>
> wrote:
>
>> Py example doesn't work on my cluster with following error:
>>
>> 15/10/05 09:44:00 DEBUG protocol.UplinkReader: Handling uplink command: -1
>> 15/10/05 09:44:00 ERROR protocol.UplinkReader: java.lang.Exception:
>> Bad command code: -1
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>
>>
>>
>>
>> On Thu, Oct 8, 2015 at 3:52 AM, Martin Illecker <mi...@apache.org>
>> wrote:
>> > Hello Vincenzo,
>> >
>> > this issue seems to be related to the LocalBSPRunner.
>> > I believe the Python example should work in Pseudo Distributed Mode [1].
>> >
>> > The error message "Traceback (most recent call last):" is caused by
>> Python
>> > because the file BSPPeer.py cannot be found by the interpreter.
>> >
>> > You should also have the following error message in the log file
>> >
>> hama/logs/tasklogs/job_localrunner_0001/attempt_localrunner_0001_000000_0.log
>> >
>> > Traceback (most recent call last):
>> >
>> > File "/..../BSPRunner.py", line 8, in <module>
>> >
>> > from BSPPeer import BSPPeer
>> >
>> > ImportError: No module named 'BSPPeer'
>> >
>> > This indicates that the parameter -cachefiles /tmp/PyStreaming/*.py is
>> not
>> > properly working in local mode.
>> >
>> > I hope you can run the Python example in Pseudo Distributed Mode in the
>> > meantime.
>> >
>> > The Hama streaming guide also suggests to run the Python example in
>> Pseudo
>> > Distributed Mode [2].
>> >
>> > Best regards
>> > Martin
>> >
>> > [1] https://wiki.apache.org/hama/GettingStarted#Pseudo_Distributed_Mode
>> > [2] http://wiki.apache.org/hama/HamaStreaming
>> >
>> > 2015-10-07 12:51 GMT+02:00 Martin Illecker <mi...@apache.org>:
>> >
>> >> Hello Vincenzo,
>> >>
>> >> I'm sorry for the inconvenience!
>> >>
>> >> I'm investigating this issue and will provide a solution until tomorrow.
>> >>
>> >> Best regards
>> >> Martin
>> >>
>> >> 2015-10-07 12:41 GMT+02:00 Vincenzo De Leo <na...@gmail.com>:
>> >>
>> >>> Thank You Edward,
>> >>> can you explain to me how can I use your solution? I have tried with:
>> >>>
>> >>> - bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>> >>> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>> >>> -inputformat
>> >>> org.apache.hama.bsp.NullInputFormat -outputformat
>> >>> org.apache.hama.bsp.SequenceFileOutputFormat -program
>> >>> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> >>>
>> >>> but I received the same old error messages... Do I have to install some
>> >>> patch and how can I eventually do this?
>> >>>
>> >>> Vincenzo
>> >>>
>> >>> On Mon, Oct 5, 2015 at 2:41 AM, Edward J. Yoon <ed...@apache.org>
>> >>> wrote:
>> >>>
>> >>> > Hi Vincenzo,
>> >>> >
>> >>> > I just found some code changes related to this problem at
>> >>> > https://issues.apache.org/jira/browse/HAMA-839, and I was able to
>> run
>> >>> > Python program by adding the in/output formatters like: -inputformat
>> >>> > org.apache.hama.bsp.NullInputFormat -outputformat
>> >>> > org.apache.hama.bsp.SequenceFileOutputFormat.
>> >>> >
>> >>> > I'll check more and update the Wiki page. Sorry for inconvenient!
>> >>> >
>> >>> >
>> >>> > On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <
>> edwardyoon@apache.org>
>> >>> > wrote:
>> >>> > > I can look at it closely next week. BTW, should input be provided?
>> >>> > >
>> >>> > > On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
>> >>> > > <vi...@linkalab.it> wrote:
>> >>> > >> Thank you for your kind reply; this is the output of the K-Means
>> >>> > example:
>> >>> > >>
>> >>> > >> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>> >>> > python3.2
>> >>> > >> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>> -program
>> >>> > >> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
>> >>> > >> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load
>> >>> > native-hadoop
>> >>> > >> library for your platform... using builtin-java classes where
>> >>> applicable
>> >>> > >> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
>> >>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name
>> is
>> >>> > >> deprecated. Instead, use fs.defaultFS
>> >>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is
>> >>> > deprecated.
>> >>> > >> Instead, use mapreduce.job.user.name
>> >>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> >>> > :an
>> >>> > >> attempt to override final parameter:
>> >>> > >> mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> >>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> >>> > :an
>> >>> > >> attempt to override final parameter:
>> >>> > >> mapreduce.job.end-notification.max.attempts; Ignoring.
>> >>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is
>> >>> > deprecated.
>> >>> > >> Instead, use mapreduce.job.user.name
>> >>> > >> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job:
>> >>> > job_localrunner_0001
>> >>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation:
>> >>> > mapred.cache.localFiles
>> >>> > >> is deprecated. Instead, use mapreduce.job.cache.local.files
>> >>> > >> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new
>> barrier
>> >>> for
>> >>> > 2
>> >>> > >> tasks!
>> >>> > >> java.lang.NumberFormatException: For input string: "Traceback
>> (most
>> >>> > recent
>> >>> > >> call last):"
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >>> > >> at java.lang.Integer.parseInt(Integer.java:580)
>> >>> > >> at java.lang.Integer.parseInt(Integer.java:615)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >>> > >> java.lang.NumberFormatException: For input string: "Traceback
>> (most
>> >>> > recent
>> >>> > >> call last):"
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >>> > >> at java.lang.Integer.parseInt(Integer.java:580)
>> >>> > >> at java.lang.Integer.parseInt(Integer.java:615)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader:
>> java.lang.Exception:
>> >>> Bad
>> >>> > >> command code: -2
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>
>> >>> > >> Exception in thread "pipe-uplink-handler"
>> java.lang.RuntimeException:
>> >>> > >> java.lang.Exception: Bad command code: -2
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >>> > >> Caused by: java.lang.Exception: Bad command code: -2
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >> java.util.concurrent.BrokenBarrierException
>> >>> > >> at
>> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >>> > >> at
>> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >> at java.lang.Thread.run(Thread.java:745)
>> >>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader:
>> java.lang.Exception:
>> >>> Bad
>> >>> > >> command code: -2
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>
>> >>> > >> java.util.concurrent.BrokenBarrierException
>> >>> > >> at
>> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >>> > >> at
>> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >> at java.lang.Thread.run(Thread.java:745)
>> >>> > >> Exception in thread "pipe-uplink-handler"
>> java.lang.RuntimeException:
>> >>> > >> java.lang.Exception: Bad command code: -2
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >>> > >> Caused by: java.lang.Exception: Bad command code: -2
>> >>> > >> at
>> >>> > >>
>> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> >>> > execution!
>> >>> > >> java.io.IOException: Stream closed
>> >>> > >> at
>> >>> > >>
>> >>>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >> at
>> >>> > >>
>> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >> at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >> at java.lang.Thread.run(Thread.java:745)
>> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> >>> > execution!
>> >>> > >> java.io.IOException: Stream closed
>> >>> > >> at
>> >>> > >>
>> >>>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >> at
>> >>> > >>
>> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >> at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >> at java.lang.Thread.run(Thread.java:745)
>> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up
>> after
>> >>> bsp
>> >>> > >> execution.
>> >>> > >> java.io.IOException: Stream closed
>> >>> > >> at
>> >>> > >>
>> >>>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >> at
>> >>> > >>
>> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >> at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >>> > >> at
>> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >> at java.lang.Thread.run(Thread.java:745)
>> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> >>> > execution!
>> >>> > >> java.util.concurrent.ExecutionException: java.io.IOException:
>> Stream
>> >>> > closed
>> >>> > >> at
>> >>> java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> >>> > >> at
>> java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> >>> > >> at java.lang.Thread.run(Thread.java:745)
>> >>> > >> Caused by: java.io.IOException: Stream closed
>> >>> > >> at
>> >>> > >>
>> >>>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >> at
>> >>> > >>
>> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >> at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >> ... 1 more
>> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up
>> after
>> >>> bsp
>> >>> > >> execution.
>> >>> > >> java.io.IOException: Stream closed
>> >>> > >> at
>> >>> > >>
>> >>>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >> at
>> >>> > >>
>> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >> at
>> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >> at
>> java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >>> > >> at
>> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >> at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >> at
>> >>> > >>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >> at java.lang.Thread.run(Thread.java:745)
>> >>> > >> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
>> >>> > >>
>> >>> > >> Vincenzo
>> >>> > >>
>> >>> > >> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <
>> >>> edwardyoon@apache.org>
>> >>> > >> wrote:
>> >>> > >>
>> >>> > >>> It looks like a bug of input reader. Can you try to run K-Means
>> >>> > example?
>> >>> > >>>
>> >>> > >>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
>> >>> > >>> <na...@gmail.com> wrote:
>> >>> > >>> > Hi to all,
>> >>> > >>> > this is my first mail to this mailing list so please have
>> patience
>> >>> > if I
>> >>> > >>> > make some bad choice in the format.
>> >>> > >>> >
>> >>> > >>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor
>> T7500
>> >>> > with 2
>> >>> > >>> GB
>> >>> > >>> > of RAM.
>> >>> > >>> >
>> >>> > >>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
>> >>> > >>> Hama-0.7.0
>> >>> > >>> > as you can see from the following lines:
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> >>
>> >>> > >>> >> > $ hadoop version
>> >>> > >>> >> > Hadoop 2.7.1
>> >>> > >>> >> > Subversion
>> https://git-wip-us.apache.org/repos/asf/hadoop.git
>> >>> -r
>> >>> > >>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
>> >>> > >>> >> > Compiled by jenkins on 2015-06-29T06:04Z
>> >>> > >>> >> > Compiled with protoc 2.5.0
>> >>> > >>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
>> >>> > >>> >> > This command was run using
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> >> > $ java -version
>> >>> > >>> >> > java version "1.8.0_60"
>> >>> > >>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
>> >>> > >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed
>> mode)
>> >>> > >>> >> >
>> >>> > >>> >
>> >>> > >>> > I am able to properly run the basic Hadoop and Hama examples
>> like
>> >>> the
>> >>> > >>> > following:
>> >>> > >>> > bin/hadoop jar
>> >>> > share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
>> >>> > >>> > grep input output 'dfs[a-z.]+'
>> >>> > >>> > bin/hama jar hama-examples-0.7.0.jar pi
>> >>> > >>> > My problem is that I receive the following error message when I
>> >>> try
>> >>> > to
>> >>> > >>> run
>> >>> > >>> > the HelloWorld Hama Streaming example with this instruction:
>> >>> > >>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter
>> python3.2
>> >>> > >>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>> >>> -program
>> >>> > >>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> >>> > >>> > The default python interpreters of my OS were python-2.7 and
>> >>> > python-3.4;
>> >>> > >>> > since I had problems with this example I also tried to install
>> >>> > python-3.2
>> >>> > >>> > with the following instructions but it didn't solved the
>> problem:
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> >>
>> >>> > >>> >> > sudo apt-get install software-properties-common
>> >>> > >>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
>> >>> > >>> >> > sudo apt-get update
>> >>> > >>> >> > sudo apt-get install python3.2
>> >>> > >>> >> >
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> > The installed python version is 3.2.6 as you can see from the
>> >>> > following
>> >>> > >>> > lines:
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> >>
>> >>> > >>> >> > $ python3.2
>> >>> > >>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
>> >>> > >>> >> > [GCC 4.8.2] on linux2
>> >>> > >>> >> > Type "help", "copyright", "credits" or "license" for more
>> >>> > information.
>> >>> > >>> >> > >>>
>> >>> > >>> >> >
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> > The error message is the following (I am working in local mode
>> so
>> >>> I
>> >>> > >>> didn't
>> >>> > >>> > run bin/start-bspd.sh):
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> >>
>> >>> > >>> >>
>> >>> > >>> >> >
>> >>> > >>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2
>> -interpreter
>> >>> > >>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output
>> >>> > /tmp/pystream-out/
>> >>> > >>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs
>> HelloWorldBSP
>> >>> > >>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
>> >>> > >>> > native-hadoop library for your platform... using builtin-java
>> >>> classes
>> >>> > >>> where
>> >>> > >>> > applicable
>> >>> > >>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
>> >>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation:
>> >>> fs.default.name
>> >>> > is
>> >>> > >>> > deprecated. Instead, use fs.defaultFS
>> >>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name
>> is
>> >>> > >>> > deprecated. Instead, use mapreduce.job.user.name
>> >>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> >>> > >>> :an
>> >>> > >>> > attempt to override final parameter:
>> >>> > >>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> >>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> >>> > >>> :an
>> >>> > >>> > attempt to override final parameter:
>> >>> > >>> > mapreduce.job.end-notification.max.attempts; Ignoring.
>> >>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name
>> is
>> >>> > >>> > deprecated. Instead, use mapreduce.job.user.name
>> >>> > >>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
>> >>> > >>> > job_localrunner_0001
>> >>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
>> >>> > >>> > mapred.cache.localFiles is deprecated. Instead, use
>> >>> > >>> > mapreduce.job.cache.local.files
>> >>> > >>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new
>> >>> > barrier
>> >>> > >>> for
>> >>> > >>> > 2 tasks!
>> >>> > >>> >> > java.lang.NumberFormatException: For input string:
>> "Traceback
>> >>> > (most
>> >>> > >>> > recent call last):"
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >>> > >>> >> > java.lang.NumberFormatException: For input string:
>> "Traceback
>> >>> > (most
>> >>> > >>> > recent call last):"
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>> >>> > java.lang.Exception:
>> >>> > >>> Bad
>> >>> > >>> > command code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>> >> >
>> >>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>> >>> > java.lang.Exception:
>> >>> > >>> Bad
>> >>> > >>> > command code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>> >> >
>> >>> > >>> >> > java.util.concurrent.BrokenBarrierException
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >>> > >>> >> > at
>> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> > >>> >> > Exception in thread "pipe-uplink-handler"
>> >>> > java.lang.RuntimeException:
>> >>> > >>> > java.lang.Exception: Bad command code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>> >> > java.util.concurrent.BrokenBarrierException
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >>> > >>> >> > at
>> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> > >>> >> > Exception in thread "pipe-uplink-handler"
>> >>> > java.lang.RuntimeException:
>> >>> > >>> > java.lang.Exception: Bad command code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>> >>> BSP
>> >>> > >>> > execution!
>> >>> > >>> >> > java.io.IOException: Stream closed
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >>> >> > at
>> >>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >>> > >>> >> > at
>> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>> >>> BSP
>> >>> > >>> > execution!
>> >>> > >>> >> > java.lang.InterruptedException: java.lang.Exception: Bad
>> >>> command
>> >>> > code:
>> >>> > >>> > -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>> >> >
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> >>> > >>> >> > at
>> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning
>> up
>> >>> > after
>> >>> > >>> bsp
>> >>> > >>> > execution.
>> >>> > >>> >> > java.io.IOException: Stream closed
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >>> >> > at
>> >>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >>> > >>> >> > at
>> >>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning
>> up
>> >>> > after
>> >>> > >>> bsp
>> >>> > >>> > execution.
>> >>> > >>> >> > java.io.IOException: Stream closed
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >>> > >>> >> > at
>> >>> > >>> >
>> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >>> > >>> >> > at
>> >>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >>> > >>> >> > at
>> >>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>> >>> BSP
>> >>> > >>> > execution!
>> >>> > >>> >> > java.util.concurrent.ExecutionException:
>> >>> > >>> > java.lang.InterruptedException: java.lang.Exception: Bad
>> command
>> >>> > code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>> >> >
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> >>> > >>> >> > Caused by: java.lang.InterruptedException:
>> java.lang.Exception:
>> >>> > Bad
>> >>> > >>> > command code: -2
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >>> > >>> >> >
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> >>> > >>> >> > at
>> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >>> > >>> >> > at
>> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >>> > >>> >> > at
>> >>> > >>> >
>> >>> > >>>
>> >>> >
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >>> > >>> >> > ... 1 more
>> >>> > >>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
>> >>> > >>> >> >
>> >>> > >>> >
>> >>> > >>> >
>> >>> > >>> > I tried to searching for these error messages with google but I
>> >>> > didn't
>> >>> > >>> find
>> >>> > >>> > any solution.
>> >>> > >>> >
>> >>> > >>> > Does anyone of you have any idea of how can I solve this
>> problem?
>> >>> > >>> >
>> >>> > >>> > Any help is appreciated :)
>> >>> > >>> >
>> >>> > >>> > Thank you
>> >>> > >>> > Best
>> >>> > >>> >
>> >>> > >>> > Vincenzo
>> >>> > >>>
>> >>> > >>>
>> >>> > >>>
>> >>> > >>> --
>> >>> > >>> Best Regards, Edward J. Yoon
>> >>> > >>>
>> >>> > >>
>> >>> > >>
>> >>> > >>
>> >>> > >> --
>> >>> > >>
>> >>> > >>
>> >>> >
>> >>>
>> -----------------------------------------------------------------------------
>> >>> > >> Dr. Vincenzo De Leo, PhD
>> >>> > >> Research and Development @ Linkalab
>> >>> > >>
>> >>> >
>> >>>
>> -----------------------------------------------------------------------------
>> >>> > >> Home Page: http://www.linkalab.it/people/vincenzo.deleo
>> >>> > >> e-mail: vincenzo.deleo@linkalab.it
>> >>> > >> Phone: (+39)0706753140
>> >>> > >> Mobile: (+39)3406673544
>> >>> > >> Skype contact: vincenzo.deleo77
>> >>> > >> GTalk contact: nagatobimaru
>> >>> > >>
>> >>> >
>> >>>
>> -----------------------------------------------------------------------------
>> >>> > >> Linkalab (http://www.linkalab.it)
>> >>> > >> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
>> >>> > >> Via San Benedetto, 88
>> >>> > >> 09129 Cagliari (CA), ITALY
>> >>> > >> P. IVA: 03148020922
>> >>> > >> Phone: (+39) 0707966021
>> >>> > >> FAX: (+39) 0707962000
>> >>> > >>
>> >>> >
>> >>>
>> -----------------------------------------------------------------------------
>> >>> > >
>> >>> > >
>> >>> > >
>> >>> > > --
>> >>> > > Best Regards, Edward J. Yoon
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Best Regards, Edward J. Yoon
>> >>> >
>> >>>
>> >>
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>>
>
>
>
> --
>
> -----------------------------------------------------------------------------
> Dr. Vincenzo De Leo, PhD
> Research and Development @ Linkalab
> -----------------------------------------------------------------------------
> Home Page: http://www.linkalab.it/people/vincenzo.deleo
> e-mail: vincenzo.deleo@linkalab.it
> Phone: (+39)0706753140
> Mobile: (+39)3406673544
> Skype contact: vincenzo.deleo77
> GTalk contact: nagatobimaru
> -----------------------------------------------------------------------------
> Linkalab (http://www.linkalab.it)
> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
> Via San Benedetto, 88
> 09129 Cagliari (CA), ITALY
> P. IVA: 03148020922
> Phone: (+39) 0707966021
> FAX: (+39) 0707962000
> -----------------------------------------------------------------------------
--
Best Regards, Edward J. Yoon
Re: hama streaming problem with hello world example
Posted by Vincenzo De Leo <vi...@linkalab.it>.
Hello Edward,
I'm sorry but in this month I had to work at other stuff so I had the time
to come back to try solve this problem only now.
I followed your hints and started working in the Pseudo Distributed Mode
but I still receive some new error messages.
With respect to the original information that I gave you I modified the Sun
Java JDK version (1.7.0_55 and not 1.8) python version (3.2.3 and not 3.4),
hama version (now I am using hama-trunk and no more hama-0.7.0) and
installed the hama-patch 805 (https://issues.apache.org/jira/browse/HAMA-805)
also if this last return to me some error message so I am not exactly sure
to have installed properly; these error messages are described below:
I copied HAMA-805.patch in the same directory of hama-trunk then run
patch -p1 < HAMA-805.patch
and received these messages:
hadoop_admin@tora-VirtualBox:~/hama-trunk$ patch -p1 < HAMA-805.patch
can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index:
core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
|===================================================================
|---
core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
(revision 1526180)
|+++
core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
(working copy)
--------------------------
File to patch:
/home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
patching file
/home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
Hunk #1 FAILED at 35.
Hunk #2 succeeded at 66 with fuzz 2 (offset 5 lines).
Hunk #3 FAILED at 272.
2 out of 3 hunks FAILED -- saving rejects to file
/home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java.rej
can't find file to patch at input line 45
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index:
core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
|===================================================================
|---
core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
(revision 1526180)
|+++
core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
(working copy)
--------------------------
File to patch:
/home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
patching file
/home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
Hunk #1 succeeded at 40 with fuzz 1 (offset 1 line).
Hunk #2 FAILED at 78.
Hunk #3 FAILED at 117.
2 out of 3 hunks FAILED -- saving rejects to file
/home/hadoop_admin/hama-trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java.rej
In any case I restarted the bspmaster and groomserver and submit again the
same job:
- hadoop_admin@tora-VirtualBox:~/hama-trunk$ bin/hama pipes -streaming
true -bspTasks 2 -interpreter python3.2 -cachefiles /tmp/PyStreaming/*.py
-output /tmp/pystream-out/ -program /tmp/PyStreaming/BSPRunner.py
-programArgs HelloWorldBSP
This time it seems that hama have properly read-access to /tmp/PyStreaming
but it seems that it has no write-access to /tmp/pystream-out/:
15/11/24 00:01:42 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable
15/11/24 00:01:44 INFO ipc.Server: Starting Socket Reader #1 for port 61002
15/11/24 00:01:44 INFO ipc.Server: IPC Server Responder: starting
15/11/24 00:01:44 INFO ipc.Server: IPC Server listener on 61002: starting
15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 1 on 61002: starting
15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 2 on 61002: starting
15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 3 on 61002: starting
15/11/24 00:01:44 INFO message.HamaMessageManagerImpl: BSPPeer
address:localhost port:61002
15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 4 on 61002: starting
15/11/24 00:01:44 INFO ipc.Server: IPC Server handler 0 on 61002: starting
15/11/24 00:01:44 FATAL bsp.GroomServer: Error running child
java.io.FileNotFoundException: /tmp/pystream-out/part-00001 (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:212)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:222)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:209)
at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:305)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:294)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:326)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.(ChecksumFileSystem.java:393)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:776)
at org.apache.hama.bsp.TextOutputFormat.getRecordWriter(TextOutputFormat.java:112)
at org.apache.hama.bsp.BSPPeerImpl.initializeIO(BSPPeerImpl.java:329)
at org.apache.hama.bsp.BSPPeerImpl.(BSPPeerImpl.java:184)
at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1251)
java.io.FileNotFoundException: /tmp/pystream-out/part-00001 (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:212)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:222)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:209)
at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:305)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:294)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:326)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.(ChecksumFileSystem.java:393)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:776)
at org.apache.hama.bsp.TextOutputFormat.getRecordWriter(TextOutputFormat.java:112)
at org.apache.hama.bsp.BSPPeerImpl.initializeIO(BSPPeerImpl.java:329)
at org.apache.hama.bsp.BSPPeerImpl.(BSPPeerImpl.java:184)
at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1251)
I also tryied to set full permissions to /tmp/pystream-out with hadoop fs
-chmod 777 /tmp/pystream-out:
hadoop_admin@tora-VirtualBox:~/hama-trunk$ hadoop fs -ls /tmp
15/11/24 00:13:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Found 3 items
drwxr-xr-x - hadoop_admin supergroup 0 2015-11-23 22:38
/tmp/PyStreaming
drwxr-xr-x - hadoop_admin supergroup 0 2015-11-23 22:40
/tmp/hadoop-tora
drwxrwxrwx - hadoop_admin supergroup 0 2015-11-23 23:17
/tmp/pystream-out
but the error message did not changed.
Any help is appreciated,
thank you very much,
bye!
Vincenzo
On Thu, Oct 8, 2015 at 3:15 AM, Edward J. Yoon <ed...@apache.org>
wrote:
> Py example doesn't work on my cluster with following error:
>
> 15/10/05 09:44:00 DEBUG protocol.UplinkReader: Handling uplink command: -1
> 15/10/05 09:44:00 ERROR protocol.UplinkReader: java.lang.Exception:
> Bad command code: -1
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>
>
>
>
> On Thu, Oct 8, 2015 at 3:52 AM, Martin Illecker <mi...@apache.org>
> wrote:
> > Hello Vincenzo,
> >
> > this issue seems to be related to the LocalBSPRunner.
> > I believe the Python example should work in Pseudo Distributed Mode [1].
> >
> > The error message "Traceback (most recent call last):" is caused by
> Python
> > because the file BSPPeer.py cannot be found by the interpreter.
> >
> > You should also have the following error message in the log file
> >
> hama/logs/tasklogs/job_localrunner_0001/attempt_localrunner_0001_000000_0.log
> >
> > Traceback (most recent call last):
> >
> > File "/..../BSPRunner.py", line 8, in <module>
> >
> > from BSPPeer import BSPPeer
> >
> > ImportError: No module named 'BSPPeer'
> >
> > This indicates that the parameter -cachefiles /tmp/PyStreaming/*.py is
> not
> > properly working in local mode.
> >
> > I hope you can run the Python example in Pseudo Distributed Mode in the
> > meantime.
> >
> > The Hama streaming guide also suggests to run the Python example in
> Pseudo
> > Distributed Mode [2].
> >
> > Best regards
> > Martin
> >
> > [1] https://wiki.apache.org/hama/GettingStarted#Pseudo_Distributed_Mode
> > [2] http://wiki.apache.org/hama/HamaStreaming
> >
> > 2015-10-07 12:51 GMT+02:00 Martin Illecker <mi...@apache.org>:
> >
> >> Hello Vincenzo,
> >>
> >> I'm sorry for the inconvenience!
> >>
> >> I'm investigating this issue and will provide a solution until tomorrow.
> >>
> >> Best regards
> >> Martin
> >>
> >> 2015-10-07 12:41 GMT+02:00 Vincenzo De Leo <na...@gmail.com>:
> >>
> >>> Thank You Edward,
> >>> can you explain to me how can I use your solution? I have tried with:
> >>>
> >>> - bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> >>> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> >>> -inputformat
> >>> org.apache.hama.bsp.NullInputFormat -outputformat
> >>> org.apache.hama.bsp.SequenceFileOutputFormat -program
> >>> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> >>>
> >>> but I received the same old error messages... Do I have to install some
> >>> patch and how can I eventually do this?
> >>>
> >>> Vincenzo
> >>>
> >>> On Mon, Oct 5, 2015 at 2:41 AM, Edward J. Yoon <ed...@apache.org>
> >>> wrote:
> >>>
> >>> > Hi Vincenzo,
> >>> >
> >>> > I just found some code changes related to this problem at
> >>> > https://issues.apache.org/jira/browse/HAMA-839, and I was able to
> run
> >>> > Python program by adding the in/output formatters like: -inputformat
> >>> > org.apache.hama.bsp.NullInputFormat -outputformat
> >>> > org.apache.hama.bsp.SequenceFileOutputFormat.
> >>> >
> >>> > I'll check more and update the Wiki page. Sorry for inconvenient!
> >>> >
> >>> >
> >>> > On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <
> edwardyoon@apache.org>
> >>> > wrote:
> >>> > > I can look at it closely next week. BTW, should input be provided?
> >>> > >
> >>> > > On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
> >>> > > <vi...@linkalab.it> wrote:
> >>> > >> Thank you for your kind reply; this is the output of the K-Means
> >>> > example:
> >>> > >>
> >>> > >> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> >>> > python3.2
> >>> > >> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> -program
> >>> > >> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
> >>> > >> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load
> >>> > native-hadoop
> >>> > >> library for your platform... using builtin-java classes where
> >>> applicable
> >>> > >> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
> >>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name
> is
> >>> > >> deprecated. Instead, use fs.defaultFS
> >>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is
> >>> > deprecated.
> >>> > >> Instead, use mapreduce.job.user.name
> >>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
> >>> > >>
> >>> >
> >>>
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> >>> > :an
> >>> > >> attempt to override final parameter:
> >>> > >> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> >>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
> >>> > >>
> >>> >
> >>>
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> >>> > :an
> >>> > >> attempt to override final parameter:
> >>> > >> mapreduce.job.end-notification.max.attempts; Ignoring.
> >>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is
> >>> > deprecated.
> >>> > >> Instead, use mapreduce.job.user.name
> >>> > >> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job:
> >>> > job_localrunner_0001
> >>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation:
> >>> > mapred.cache.localFiles
> >>> > >> is deprecated. Instead, use mapreduce.job.cache.local.files
> >>> > >> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new
> barrier
> >>> for
> >>> > 2
> >>> > >> tasks!
> >>> > >> java.lang.NumberFormatException: For input string: "Traceback
> (most
> >>> > recent
> >>> > >> call last):"
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >>> > >> at java.lang.Integer.parseInt(Integer.java:580)
> >>> > >> at java.lang.Integer.parseInt(Integer.java:615)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >>> > >> java.lang.NumberFormatException: For input string: "Traceback
> (most
> >>> > recent
> >>> > >> call last):"
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >>> > >> at java.lang.Integer.parseInt(Integer.java:580)
> >>> > >> at java.lang.Integer.parseInt(Integer.java:615)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader:
> java.lang.Exception:
> >>> Bad
> >>> > >> command code: -2
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>
> >>> > >> Exception in thread "pipe-uplink-handler"
> java.lang.RuntimeException:
> >>> > >> java.lang.Exception: Bad command code: -2
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >>> > >> Caused by: java.lang.Exception: Bad command code: -2
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >> java.util.concurrent.BrokenBarrierException
> >>> > >> at
> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >>> > >> at
> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >> at java.lang.Thread.run(Thread.java:745)
> >>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader:
> java.lang.Exception:
> >>> Bad
> >>> > >> command code: -2
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>
> >>> > >> java.util.concurrent.BrokenBarrierException
> >>> > >> at
> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >>> > >> at
> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >> at java.lang.Thread.run(Thread.java:745)
> >>> > >> Exception in thread "pipe-uplink-handler"
> java.lang.RuntimeException:
> >>> > >> java.lang.Exception: Bad command code: -2
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >>> > >> Caused by: java.lang.Exception: Bad command code: -2
> >>> > >> at
> >>> > >>
> >>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> >>> > execution!
> >>> > >> java.io.IOException: Stream closed
> >>> > >> at
> >>> > >>
> >>>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >> at
> >>> > >>
> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >> at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >> at java.lang.Thread.run(Thread.java:745)
> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> >>> > execution!
> >>> > >> java.io.IOException: Stream closed
> >>> > >> at
> >>> > >>
> >>>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >> at
> >>> > >>
> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >> at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >> at java.lang.Thread.run(Thread.java:745)
> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up
> after
> >>> bsp
> >>> > >> execution.
> >>> > >> java.io.IOException: Stream closed
> >>> > >> at
> >>> > >>
> >>>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >> at
> >>> > >>
> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >> at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >>> > >> at
> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >> at java.lang.Thread.run(Thread.java:745)
> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> >>> > execution!
> >>> > >> java.util.concurrent.ExecutionException: java.io.IOException:
> Stream
> >>> > closed
> >>> > >> at
> >>> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >>> > >> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> >>> > >> at java.lang.Thread.run(Thread.java:745)
> >>> > >> Caused by: java.io.IOException: Stream closed
> >>> > >> at
> >>> > >>
> >>>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >> at
> >>> > >>
> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >> at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >> ... 1 more
> >>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up
> after
> >>> bsp
> >>> > >> execution.
> >>> > >> java.io.IOException: Stream closed
> >>> > >> at
> >>> > >>
> >>>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >> at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >> at
> >>> > >>
> >>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >> at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >> at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >>> > >> at
> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >> at
> >>> > >>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >> at java.lang.Thread.run(Thread.java:745)
> >>> > >> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
> >>> > >>
> >>> > >> Vincenzo
> >>> > >>
> >>> > >> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <
> >>> edwardyoon@apache.org>
> >>> > >> wrote:
> >>> > >>
> >>> > >>> It looks like a bug of input reader. Can you try to run K-Means
> >>> > example?
> >>> > >>>
> >>> > >>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
> >>> > >>> <na...@gmail.com> wrote:
> >>> > >>> > Hi to all,
> >>> > >>> > this is my first mail to this mailing list so please have
> patience
> >>> > if I
> >>> > >>> > make some bad choice in the format.
> >>> > >>> >
> >>> > >>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor
> T7500
> >>> > with 2
> >>> > >>> GB
> >>> > >>> > of RAM.
> >>> > >>> >
> >>> > >>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
> >>> > >>> Hama-0.7.0
> >>> > >>> > as you can see from the following lines:
> >>> > >>> >
> >>> > >>> >
> >>> > >>> >>
> >>> > >>> >> > $ hadoop version
> >>> > >>> >> > Hadoop 2.7.1
> >>> > >>> >> > Subversion
> https://git-wip-us.apache.org/repos/asf/hadoop.git
> >>> -r
> >>> > >>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
> >>> > >>> >> > Compiled by jenkins on 2015-06-29T06:04Z
> >>> > >>> >> > Compiled with protoc 2.5.0
> >>> > >>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
> >>> > >>> >> > This command was run using
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
> >>> > >>> >
> >>> > >>> >
> >>> > >>> >> > $ java -version
> >>> > >>> >> > java version "1.8.0_60"
> >>> > >>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> >>> > >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed
> mode)
> >>> > >>> >> >
> >>> > >>> >
> >>> > >>> > I am able to properly run the basic Hadoop and Hama examples
> like
> >>> the
> >>> > >>> > following:
> >>> > >>> > bin/hadoop jar
> >>> > share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
> >>> > >>> > grep input output 'dfs[a-z.]+'
> >>> > >>> > bin/hama jar hama-examples-0.7.0.jar pi
> >>> > >>> > My problem is that I receive the following error message when I
> >>> try
> >>> > to
> >>> > >>> run
> >>> > >>> > the HelloWorld Hama Streaming example with this instruction:
> >>> > >>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter
> python3.2
> >>> > >>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> >>> -program
> >>> > >>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> >>> > >>> > The default python interpreters of my OS were python-2.7 and
> >>> > python-3.4;
> >>> > >>> > since I had problems with this example I also tried to install
> >>> > python-3.2
> >>> > >>> > with the following instructions but it didn't solved the
> problem:
> >>> > >>> >
> >>> > >>> >
> >>> > >>> >>
> >>> > >>> >> > sudo apt-get install software-properties-common
> >>> > >>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
> >>> > >>> >> > sudo apt-get update
> >>> > >>> >> > sudo apt-get install python3.2
> >>> > >>> >> >
> >>> > >>> >
> >>> > >>> >
> >>> > >>> > The installed python version is 3.2.6 as you can see from the
> >>> > following
> >>> > >>> > lines:
> >>> > >>> >
> >>> > >>> >
> >>> > >>> >>
> >>> > >>> >> > $ python3.2
> >>> > >>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
> >>> > >>> >> > [GCC 4.8.2] on linux2
> >>> > >>> >> > Type "help", "copyright", "credits" or "license" for more
> >>> > information.
> >>> > >>> >> > >>>
> >>> > >>> >> >
> >>> > >>> >
> >>> > >>> >
> >>> > >>> > The error message is the following (I am working in local mode
> so
> >>> I
> >>> > >>> didn't
> >>> > >>> > run bin/start-bspd.sh):
> >>> > >>> >
> >>> > >>> >
> >>> > >>> >>
> >>> > >>> >>
> >>> > >>> >> >
> >>> > >>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2
> -interpreter
> >>> > >>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output
> >>> > /tmp/pystream-out/
> >>> > >>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs
> HelloWorldBSP
> >>> > >>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
> >>> > >>> > native-hadoop library for your platform... using builtin-java
> >>> classes
> >>> > >>> where
> >>> > >>> > applicable
> >>> > >>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
> >>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation:
> >>> fs.default.name
> >>> > is
> >>> > >>> > deprecated. Instead, use fs.defaultFS
> >>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name
> is
> >>> > >>> > deprecated. Instead, use mapreduce.job.user.name
> >>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> >>> > >>> >
> >>> >
> >>>
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> >>> > >>> :an
> >>> > >>> > attempt to override final parameter:
> >>> > >>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
> >>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> >>> > >>> >
> >>> >
> >>>
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> >>> > >>> :an
> >>> > >>> > attempt to override final parameter:
> >>> > >>> > mapreduce.job.end-notification.max.attempts; Ignoring.
> >>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name
> is
> >>> > >>> > deprecated. Instead, use mapreduce.job.user.name
> >>> > >>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
> >>> > >>> > job_localrunner_0001
> >>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
> >>> > >>> > mapred.cache.localFiles is deprecated. Instead, use
> >>> > >>> > mapreduce.job.cache.local.files
> >>> > >>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new
> >>> > barrier
> >>> > >>> for
> >>> > >>> > 2 tasks!
> >>> > >>> >> > java.lang.NumberFormatException: For input string:
> "Traceback
> >>> > (most
> >>> > >>> > recent call last):"
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >>> > >>> >> > java.lang.NumberFormatException: For input string:
> "Traceback
> >>> > (most
> >>> > >>> > recent call last):"
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
> >>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
> >>> > java.lang.Exception:
> >>> > >>> Bad
> >>> > >>> > command code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>> >> >
> >>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
> >>> > java.lang.Exception:
> >>> > >>> Bad
> >>> > >>> > command code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>> >> >
> >>> > >>> >> > java.util.concurrent.BrokenBarrierException
> >>> > >>> >> > at
> >>> > >>> >
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >>> > >>> >> > at
> >>> > >>> >
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >>> > >>> >> > at
> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> > >>> >> > Exception in thread "pipe-uplink-handler"
> >>> > java.lang.RuntimeException:
> >>> > >>> > java.lang.Exception: Bad command code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>> >> > java.util.concurrent.BrokenBarrierException
> >>> > >>> >> > at
> >>> > >>> >
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >>> > >>> >> > at
> >>> > >>> >
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >>> > >>> >> > at
> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> > >>> >> > Exception in thread "pipe-uplink-handler"
> >>> > java.lang.RuntimeException:
> >>> > >>> > java.lang.Exception: Bad command code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
> >>> BSP
> >>> > >>> > execution!
> >>> > >>> >> > java.io.IOException: Stream closed
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >>> >> > at
> >>> > >>> >
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >>> >> > at
> >>> > >>> >
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >>> >> > at
> >>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >>> > >>> >> > at
> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
> >>> BSP
> >>> > >>> > execution!
> >>> > >>> >> > java.lang.InterruptedException: java.lang.Exception: Bad
> >>> command
> >>> > code:
> >>> > >>> > -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>> >> >
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> >>> > >>> >> > at
> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning
> up
> >>> > after
> >>> > >>> bsp
> >>> > >>> > execution.
> >>> > >>> >> > java.io.IOException: Stream closed
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >>> >> > at
> >>> > >>> >
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >>> >> > at
> >>> > >>> >
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >>> >> > at
> >>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >>> > >>> >> > at
> >>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning
> up
> >>> > after
> >>> > >>> bsp
> >>> > >>> > execution.
> >>> > >>> >> > java.io.IOException: Stream closed
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> > >>> >> > at
> >>> > >>> >
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> > >>> >> > at
> >>> > >>> >
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> > >>> >> > at
> >>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >>> > >>> >> > at
> >>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
> >>> BSP
> >>> > >>> > execution!
> >>> > >>> >> > java.util.concurrent.ExecutionException:
> >>> > >>> > java.lang.InterruptedException: java.lang.Exception: Bad
> command
> >>> > code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>> >> >
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> >>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> > >>> >> > Caused by: java.lang.InterruptedException:
> java.lang.Exception:
> >>> > Bad
> >>> > >>> > command code: -2
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> > >>> >> >
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> >>> > >>> >> > at
> >>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> > >>> >> > at
> >>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> > >>> >> > at
> >>> > >>> >
> >>> > >>>
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> > >>> >> > ... 1 more
> >>> > >>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
> >>> > >>> >> >
> >>> > >>> >
> >>> > >>> >
> >>> > >>> > I tried to searching for these error messages with google but I
> >>> > didn't
> >>> > >>> find
> >>> > >>> > any solution.
> >>> > >>> >
> >>> > >>> > Does anyone of you have any idea of how can I solve this
> problem?
> >>> > >>> >
> >>> > >>> > Any help is appreciated :)
> >>> > >>> >
> >>> > >>> > Thank you
> >>> > >>> > Best
> >>> > >>> >
> >>> > >>> > Vincenzo
> >>> > >>>
> >>> > >>>
> >>> > >>>
> >>> > >>> --
> >>> > >>> Best Regards, Edward J. Yoon
> >>> > >>>
> >>> > >>
> >>> > >>
> >>> > >>
> >>> > >> --
> >>> > >>
> >>> > >>
> >>> >
> >>>
> -----------------------------------------------------------------------------
> >>> > >> Dr. Vincenzo De Leo, PhD
> >>> > >> Research and Development @ Linkalab
> >>> > >>
> >>> >
> >>>
> -----------------------------------------------------------------------------
> >>> > >> Home Page: http://www.linkalab.it/people/vincenzo.deleo
> >>> > >> e-mail: vincenzo.deleo@linkalab.it
> >>> > >> Phone: (+39)0706753140
> >>> > >> Mobile: (+39)3406673544
> >>> > >> Skype contact: vincenzo.deleo77
> >>> > >> GTalk contact: nagatobimaru
> >>> > >>
> >>> >
> >>>
> -----------------------------------------------------------------------------
> >>> > >> Linkalab (http://www.linkalab.it)
> >>> > >> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
> >>> > >> Via San Benedetto, 88
> >>> > >> 09129 Cagliari (CA), ITALY
> >>> > >> P. IVA: 03148020922
> >>> > >> Phone: (+39) 0707966021
> >>> > >> FAX: (+39) 0707962000
> >>> > >>
> >>> >
> >>>
> -----------------------------------------------------------------------------
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Best Regards, Edward J. Yoon
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Best Regards, Edward J. Yoon
> >>> >
> >>>
> >>
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
>
--
-----------------------------------------------------------------------------
Dr. Vincenzo De Leo, PhD
Research and Development @ Linkalab
-----------------------------------------------------------------------------
Home Page: http://www.linkalab.it/people/vincenzo.deleo
e-mail: vincenzo.deleo@linkalab.it
Phone: (+39)0706753140
Mobile: (+39)3406673544
Skype contact: vincenzo.deleo77
GTalk contact: nagatobimaru
-----------------------------------------------------------------------------
Linkalab (http://www.linkalab.it)
CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
Via San Benedetto, 88
09129 Cagliari (CA), ITALY
P. IVA: 03148020922
Phone: (+39) 0707966021
FAX: (+39) 0707962000
-----------------------------------------------------------------------------
Re: hama streaming problem with hello world example
Posted by "Edward J. Yoon" <ed...@apache.org>.
Py example doesn't work on my cluster with following error:
15/10/05 09:44:00 DEBUG protocol.UplinkReader: Handling uplink command: -1
15/10/05 09:44:00 ERROR protocol.UplinkReader: java.lang.Exception:
Bad command code: -1
at org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
On Thu, Oct 8, 2015 at 3:52 AM, Martin Illecker <mi...@apache.org> wrote:
> Hello Vincenzo,
>
> this issue seems to be related to the LocalBSPRunner.
> I believe the Python example should work in Pseudo Distributed Mode [1].
>
> The error message "Traceback (most recent call last):" is caused by Python
> because the file BSPPeer.py cannot be found by the interpreter.
>
> You should also have the following error message in the log file
> hama/logs/tasklogs/job_localrunner_0001/attempt_localrunner_0001_000000_0.log
>
> Traceback (most recent call last):
>
> File "/..../BSPRunner.py", line 8, in <module>
>
> from BSPPeer import BSPPeer
>
> ImportError: No module named 'BSPPeer'
>
> This indicates that the parameter -cachefiles /tmp/PyStreaming/*.py is not
> properly working in local mode.
>
> I hope you can run the Python example in Pseudo Distributed Mode in the
> meantime.
>
> The Hama streaming guide also suggests to run the Python example in Pseudo
> Distributed Mode [2].
>
> Best regards
> Martin
>
> [1] https://wiki.apache.org/hama/GettingStarted#Pseudo_Distributed_Mode
> [2] http://wiki.apache.org/hama/HamaStreaming
>
> 2015-10-07 12:51 GMT+02:00 Martin Illecker <mi...@apache.org>:
>
>> Hello Vincenzo,
>>
>> I'm sorry for the inconvenience!
>>
>> I'm investigating this issue and will provide a solution until tomorrow.
>>
>> Best regards
>> Martin
>>
>> 2015-10-07 12:41 GMT+02:00 Vincenzo De Leo <na...@gmail.com>:
>>
>>> Thank You Edward,
>>> can you explain to me how can I use your solution? I have tried with:
>>>
>>> - bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>>> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>>> -inputformat
>>> org.apache.hama.bsp.NullInputFormat -outputformat
>>> org.apache.hama.bsp.SequenceFileOutputFormat -program
>>> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>>>
>>> but I received the same old error messages... Do I have to install some
>>> patch and how can I eventually do this?
>>>
>>> Vincenzo
>>>
>>> On Mon, Oct 5, 2015 at 2:41 AM, Edward J. Yoon <ed...@apache.org>
>>> wrote:
>>>
>>> > Hi Vincenzo,
>>> >
>>> > I just found some code changes related to this problem at
>>> > https://issues.apache.org/jira/browse/HAMA-839, and I was able to run
>>> > Python program by adding the in/output formatters like: -inputformat
>>> > org.apache.hama.bsp.NullInputFormat -outputformat
>>> > org.apache.hama.bsp.SequenceFileOutputFormat.
>>> >
>>> > I'll check more and update the Wiki page. Sorry for inconvenient!
>>> >
>>> >
>>> > On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <ed...@apache.org>
>>> > wrote:
>>> > > I can look at it closely next week. BTW, should input be provided?
>>> > >
>>> > > On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
>>> > > <vi...@linkalab.it> wrote:
>>> > >> Thank you for your kind reply; this is the output of the K-Means
>>> > example:
>>> > >>
>>> > >> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>>> > python3.2
>>> > >> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>>> > >> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
>>> > >> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load
>>> > native-hadoop
>>> > >> library for your platform... using builtin-java classes where
>>> applicable
>>> > >> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
>>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
>>> > >> deprecated. Instead, use fs.defaultFS
>>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is
>>> > deprecated.
>>> > >> Instead, use mapreduce.job.user.name
>>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
>>> > >>
>>> >
>>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>>> > :an
>>> > >> attempt to override final parameter:
>>> > >> mapreduce.job.end-notification.max.retry.interval; Ignoring.
>>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
>>> > >>
>>> >
>>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>>> > :an
>>> > >> attempt to override final parameter:
>>> > >> mapreduce.job.end-notification.max.attempts; Ignoring.
>>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is
>>> > deprecated.
>>> > >> Instead, use mapreduce.job.user.name
>>> > >> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job:
>>> > job_localrunner_0001
>>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation:
>>> > mapred.cache.localFiles
>>> > >> is deprecated. Instead, use mapreduce.job.cache.local.files
>>> > >> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier
>>> for
>>> > 2
>>> > >> tasks!
>>> > >> java.lang.NumberFormatException: For input string: "Traceback (most
>>> > recent
>>> > >> call last):"
>>> > >> at
>>> > >>
>>> >
>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>> > >> at java.lang.Integer.parseInt(Integer.java:580)
>>> > >> at java.lang.Integer.parseInt(Integer.java:615)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>>> > >> java.lang.NumberFormatException: For input string: "Traceback (most
>>> > recent
>>> > >> call last):"
>>> > >> at
>>> > >>
>>> >
>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>> > >> at java.lang.Integer.parseInt(Integer.java:580)
>>> > >> at java.lang.Integer.parseInt(Integer.java:615)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception:
>>> Bad
>>> > >> command code: -2
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>
>>> > >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>>> > >> java.lang.Exception: Bad command code: -2
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>>> > >> Caused by: java.lang.Exception: Bad command code: -2
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >> java.util.concurrent.BrokenBarrierException
>>> > >> at
>>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>>> > >> at
>>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >> at java.lang.Thread.run(Thread.java:745)
>>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception:
>>> Bad
>>> > >> command code: -2
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>
>>> > >> java.util.concurrent.BrokenBarrierException
>>> > >> at
>>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>>> > >> at
>>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >> at java.lang.Thread.run(Thread.java:745)
>>> > >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>>> > >> java.lang.Exception: Bad command code: -2
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>>> > >> Caused by: java.lang.Exception: Bad command code: -2
>>> > >> at
>>> > >>
>>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> > execution!
>>> > >> java.io.IOException: Stream closed
>>> > >> at
>>> > >>
>>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>>> > >> at
>>> > >>
>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >> at java.lang.Thread.run(Thread.java:745)
>>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> > execution!
>>> > >> java.io.IOException: Stream closed
>>> > >> at
>>> > >>
>>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>>> > >> at
>>> > >>
>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >> at java.lang.Thread.run(Thread.java:745)
>>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after
>>> bsp
>>> > >> execution.
>>> > >> java.io.IOException: Stream closed
>>> > >> at
>>> > >>
>>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>>> > >> at
>>> > >>
>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>>> > >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >> at java.lang.Thread.run(Thread.java:745)
>>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> > execution!
>>> > >> java.util.concurrent.ExecutionException: java.io.IOException: Stream
>>> > closed
>>> > >> at
>>> java.util.concurrent.FutureTask.report(FutureTask.java:122)
>>> > >> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>>> > >> at java.lang.Thread.run(Thread.java:745)
>>> > >> Caused by: java.io.IOException: Stream closed
>>> > >> at
>>> > >>
>>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>>> > >> at
>>> > >>
>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >> ... 1 more
>>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after
>>> bsp
>>> > >> execution.
>>> > >> java.io.IOException: Stream closed
>>> > >> at
>>> > >>
>>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>>> > >> at
>>> > >>
>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >> at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>>> > >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >> at
>>> > >>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >> at
>>> > >>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >> at java.lang.Thread.run(Thread.java:745)
>>> > >> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
>>> > >>
>>> > >> Vincenzo
>>> > >>
>>> > >> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <
>>> edwardyoon@apache.org>
>>> > >> wrote:
>>> > >>
>>> > >>> It looks like a bug of input reader. Can you try to run K-Means
>>> > example?
>>> > >>>
>>> > >>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
>>> > >>> <na...@gmail.com> wrote:
>>> > >>> > Hi to all,
>>> > >>> > this is my first mail to this mailing list so please have patience
>>> > if I
>>> > >>> > make some bad choice in the format.
>>> > >>> >
>>> > >>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500
>>> > with 2
>>> > >>> GB
>>> > >>> > of RAM.
>>> > >>> >
>>> > >>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
>>> > >>> Hama-0.7.0
>>> > >>> > as you can see from the following lines:
>>> > >>> >
>>> > >>> >
>>> > >>> >>
>>> > >>> >> > $ hadoop version
>>> > >>> >> > Hadoop 2.7.1
>>> > >>> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git
>>> -r
>>> > >>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
>>> > >>> >> > Compiled by jenkins on 2015-06-29T06:04Z
>>> > >>> >> > Compiled with protoc 2.5.0
>>> > >>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
>>> > >>> >> > This command was run using
>>> > >>> >
>>> > >>>
>>> >
>>> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
>>> > >>> >
>>> > >>> >
>>> > >>> >> > $ java -version
>>> > >>> >> > java version "1.8.0_60"
>>> > >>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
>>> > >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>>> > >>> >> >
>>> > >>> >
>>> > >>> > I am able to properly run the basic Hadoop and Hama examples like
>>> the
>>> > >>> > following:
>>> > >>> > bin/hadoop jar
>>> > share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
>>> > >>> > grep input output 'dfs[a-z.]+'
>>> > >>> > bin/hama jar hama-examples-0.7.0.jar pi
>>> > >>> > My problem is that I receive the following error message when I
>>> try
>>> > to
>>> > >>> run
>>> > >>> > the HelloWorld Hama Streaming example with this instruction:
>>> > >>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>>> > >>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>>> -program
>>> > >>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>>> > >>> > The default python interpreters of my OS were python-2.7 and
>>> > python-3.4;
>>> > >>> > since I had problems with this example I also tried to install
>>> > python-3.2
>>> > >>> > with the following instructions but it didn't solved the problem:
>>> > >>> >
>>> > >>> >
>>> > >>> >>
>>> > >>> >> > sudo apt-get install software-properties-common
>>> > >>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
>>> > >>> >> > sudo apt-get update
>>> > >>> >> > sudo apt-get install python3.2
>>> > >>> >> >
>>> > >>> >
>>> > >>> >
>>> > >>> > The installed python version is 3.2.6 as you can see from the
>>> > following
>>> > >>> > lines:
>>> > >>> >
>>> > >>> >
>>> > >>> >>
>>> > >>> >> > $ python3.2
>>> > >>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
>>> > >>> >> > [GCC 4.8.2] on linux2
>>> > >>> >> > Type "help", "copyright", "credits" or "license" for more
>>> > information.
>>> > >>> >> > >>>
>>> > >>> >> >
>>> > >>> >
>>> > >>> >
>>> > >>> > The error message is the following (I am working in local mode so
>>> I
>>> > >>> didn't
>>> > >>> > run bin/start-bspd.sh):
>>> > >>> >
>>> > >>> >
>>> > >>> >>
>>> > >>> >>
>>> > >>> >> >
>>> > >>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>>> > >>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output
>>> > /tmp/pystream-out/
>>> > >>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>>> > >>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
>>> > >>> > native-hadoop library for your platform... using builtin-java
>>> classes
>>> > >>> where
>>> > >>> > applicable
>>> > >>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
>>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation:
>>> fs.default.name
>>> > is
>>> > >>> > deprecated. Instead, use fs.defaultFS
>>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
>>> > >>> > deprecated. Instead, use mapreduce.job.user.name
>>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>>> > >>> >
>>> >
>>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>>> > >>> :an
>>> > >>> > attempt to override final parameter:
>>> > >>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
>>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>>> > >>> >
>>> >
>>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>>> > >>> :an
>>> > >>> > attempt to override final parameter:
>>> > >>> > mapreduce.job.end-notification.max.attempts; Ignoring.
>>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
>>> > >>> > deprecated. Instead, use mapreduce.job.user.name
>>> > >>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
>>> > >>> > job_localrunner_0001
>>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
>>> > >>> > mapred.cache.localFiles is deprecated. Instead, use
>>> > >>> > mapreduce.job.cache.local.files
>>> > >>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new
>>> > barrier
>>> > >>> for
>>> > >>> > 2 tasks!
>>> > >>> >> > java.lang.NumberFormatException: For input string: "Traceback
>>> > (most
>>> > >>> > recent call last):"
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>>> > >>> >> > java.lang.NumberFormatException: For input string: "Traceback
>>> > (most
>>> > >>> > recent call last):"
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>>> > java.lang.Exception:
>>> > >>> Bad
>>> > >>> > command code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>> >> >
>>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>>> > java.lang.Exception:
>>> > >>> Bad
>>> > >>> > command code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>> >> >
>>> > >>> >> > java.util.concurrent.BrokenBarrierException
>>> > >>> >> > at
>>> > >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>>> > >>> >> > at
>>> > >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>>> > >>> >> > at
>>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>>> > >>> >> > at
>>> > >>> >
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>>> > >>> >> > Exception in thread "pipe-uplink-handler"
>>> > java.lang.RuntimeException:
>>> > >>> > java.lang.Exception: Bad command code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>> >> > java.util.concurrent.BrokenBarrierException
>>> > >>> >> > at
>>> > >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>>> > >>> >> > at
>>> > >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>>> > >>> >> > at
>>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>>> > >>> >> > at
>>> > >>> >
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>>> > >>> >> > Exception in thread "pipe-uplink-handler"
>>> > java.lang.RuntimeException:
>>> > >>> > java.lang.Exception: Bad command code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>>> BSP
>>> > >>> > execution!
>>> > >>> >> > java.io.IOException: Stream closed
>>> > >>> >> > at
>>> > >>> >
>>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>>> > >>> >> > at
>>> > >>> >
>>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >>> >> > at
>>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >>> >> > at
>>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >>> >> > at
>>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>>> > >>> >> > at
>>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>>> > >>> >> > at
>>> > >>> >
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>>> BSP
>>> > >>> > execution!
>>> > >>> >> > java.lang.InterruptedException: java.lang.Exception: Bad
>>> command
>>> > code:
>>> > >>> > -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>> >> >
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>>> > >>> >> > at
>>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>>> > >>> >> > at
>>> > >>> >
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
>>> > after
>>> > >>> bsp
>>> > >>> > execution.
>>> > >>> >> > java.io.IOException: Stream closed
>>> > >>> >> > at
>>> > >>> >
>>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>>> > >>> >> > at
>>> > >>> >
>>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >>> >> > at
>>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >>> >> > at
>>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >>> >> > at
>>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>>> > >>> >> > at
>>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>>> > >>> >> > at
>>> > >>> >
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
>>> > after
>>> > >>> bsp
>>> > >>> > execution.
>>> > >>> >> > java.io.IOException: Stream closed
>>> > >>> >> > at
>>> > >>> >
>>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>>> > >>> >> > at
>>> > >>> >
>>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> > >>> >> > at
>>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> > >>> >> > at
>>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> > >>> >> > at
>>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>>> > >>> >> > at
>>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>>> > >>> >> > at
>>> > >>> >
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>>> BSP
>>> > >>> > execution!
>>> > >>> >> > java.util.concurrent.ExecutionException:
>>> > >>> > java.lang.InterruptedException: java.lang.Exception: Bad command
>>> > code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>> >> >
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.report(FutureTask.java:122)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.get(FutureTask.java:192)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>>> > >>> >> > Caused by: java.lang.InterruptedException: java.lang.Exception:
>>> > Bad
>>> > >>> > command code: -2
>>> > >>> >> > at
>>> > >>> >
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> > >>> >> >
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>>> > >>> >> > at
>>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>>> > >>> >> > at
>>> > >>> >
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> > >>> >> > at
>>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> > >>> >> > at
>>> > >>> >
>>> > >>>
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> > >>> >> > ... 1 more
>>> > >>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
>>> > >>> >> >
>>> > >>> >
>>> > >>> >
>>> > >>> > I tried to searching for these error messages with google but I
>>> > didn't
>>> > >>> find
>>> > >>> > any solution.
>>> > >>> >
>>> > >>> > Does anyone of you have any idea of how can I solve this problem?
>>> > >>> >
>>> > >>> > Any help is appreciated :)
>>> > >>> >
>>> > >>> > Thank you
>>> > >>> > Best
>>> > >>> >
>>> > >>> > Vincenzo
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>> --
>>> > >>> Best Regards, Edward J. Yoon
>>> > >>>
>>> > >>
>>> > >>
>>> > >>
>>> > >> --
>>> > >>
>>> > >>
>>> >
>>> -----------------------------------------------------------------------------
>>> > >> Dr. Vincenzo De Leo, PhD
>>> > >> Research and Development @ Linkalab
>>> > >>
>>> >
>>> -----------------------------------------------------------------------------
>>> > >> Home Page: http://www.linkalab.it/people/vincenzo.deleo
>>> > >> e-mail: vincenzo.deleo@linkalab.it
>>> > >> Phone: (+39)0706753140
>>> > >> Mobile: (+39)3406673544
>>> > >> Skype contact: vincenzo.deleo77
>>> > >> GTalk contact: nagatobimaru
>>> > >>
>>> >
>>> -----------------------------------------------------------------------------
>>> > >> Linkalab (http://www.linkalab.it)
>>> > >> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
>>> > >> Via San Benedetto, 88
>>> > >> 09129 Cagliari (CA), ITALY
>>> > >> P. IVA: 03148020922
>>> > >> Phone: (+39) 0707966021
>>> > >> FAX: (+39) 0707962000
>>> > >>
>>> >
>>> -----------------------------------------------------------------------------
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Best Regards, Edward J. Yoon
>>> >
>>> >
>>> >
>>> > --
>>> > Best Regards, Edward J. Yoon
>>> >
>>>
>>
>>
--
Best Regards, Edward J. Yoon
Re: hama streaming problem with hello world example
Posted by Martin Illecker <mi...@apache.org>.
Hello Vincenzo,
this issue seems to be related to the LocalBSPRunner.
I believe the Python example should work in Pseudo Distributed Mode [1].
The error message "Traceback (most recent call last):" is caused by Python
because the file BSPPeer.py cannot be found by the interpreter.
You should also have the following error message in the log file
hama/logs/tasklogs/job_localrunner_0001/attempt_localrunner_0001_000000_0.log
Traceback (most recent call last):
File "/..../BSPRunner.py", line 8, in <module>
from BSPPeer import BSPPeer
ImportError: No module named 'BSPPeer'
This indicates that the parameter -cachefiles /tmp/PyStreaming/*.py is not
properly working in local mode.
I hope you can run the Python example in Pseudo Distributed Mode in the
meantime.
The Hama streaming guide also suggests to run the Python example in Pseudo
Distributed Mode [2].
Best regards
Martin
[1] https://wiki.apache.org/hama/GettingStarted#Pseudo_Distributed_Mode
[2] http://wiki.apache.org/hama/HamaStreaming
2015-10-07 12:51 GMT+02:00 Martin Illecker <mi...@apache.org>:
> Hello Vincenzo,
>
> I'm sorry for the inconvenience!
>
> I'm investigating this issue and will provide a solution until tomorrow.
>
> Best regards
> Martin
>
> 2015-10-07 12:41 GMT+02:00 Vincenzo De Leo <na...@gmail.com>:
>
>> Thank You Edward,
>> can you explain to me how can I use your solution? I have tried with:
>>
>> - bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>> -inputformat
>> org.apache.hama.bsp.NullInputFormat -outputformat
>> org.apache.hama.bsp.SequenceFileOutputFormat -program
>> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>>
>> but I received the same old error messages... Do I have to install some
>> patch and how can I eventually do this?
>>
>> Vincenzo
>>
>> On Mon, Oct 5, 2015 at 2:41 AM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>>
>> > Hi Vincenzo,
>> >
>> > I just found some code changes related to this problem at
>> > https://issues.apache.org/jira/browse/HAMA-839, and I was able to run
>> > Python program by adding the in/output formatters like: -inputformat
>> > org.apache.hama.bsp.NullInputFormat -outputformat
>> > org.apache.hama.bsp.SequenceFileOutputFormat.
>> >
>> > I'll check more and update the Wiki page. Sorry for inconvenient!
>> >
>> >
>> > On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <ed...@apache.org>
>> > wrote:
>> > > I can look at it closely next week. BTW, should input be provided?
>> > >
>> > > On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
>> > > <vi...@linkalab.it> wrote:
>> > >> Thank you for your kind reply; this is the output of the K-Means
>> > example:
>> > >>
>> > >> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>> > python3.2
>> > >> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>> > >> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
>> > >> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load
>> > native-hadoop
>> > >> library for your platform... using builtin-java classes where
>> applicable
>> > >> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
>> > >> deprecated. Instead, use fs.defaultFS
>> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is
>> > deprecated.
>> > >> Instead, use mapreduce.job.user.name
>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
>> > >>
>> >
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> > :an
>> > >> attempt to override final parameter:
>> > >> mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> > >> 15/10/01 15:06:57 WARN conf.Configuration:
>> > >>
>> >
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> > :an
>> > >> attempt to override final parameter:
>> > >> mapreduce.job.end-notification.max.attempts; Ignoring.
>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is
>> > deprecated.
>> > >> Instead, use mapreduce.job.user.name
>> > >> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job:
>> > job_localrunner_0001
>> > >> 15/10/01 15:06:57 INFO Configuration.deprecation:
>> > mapred.cache.localFiles
>> > >> is deprecated. Instead, use mapreduce.job.cache.local.files
>> > >> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier
>> for
>> > 2
>> > >> tasks!
>> > >> java.lang.NumberFormatException: For input string: "Traceback (most
>> > recent
>> > >> call last):"
>> > >> at
>> > >>
>> >
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> > >> at java.lang.Integer.parseInt(Integer.java:580)
>> > >> at java.lang.Integer.parseInt(Integer.java:615)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> > >> java.lang.NumberFormatException: For input string: "Traceback (most
>> > recent
>> > >> call last):"
>> > >> at
>> > >>
>> >
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> > >> at java.lang.Integer.parseInt(Integer.java:580)
>> > >> at java.lang.Integer.parseInt(Integer.java:615)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception:
>> Bad
>> > >> command code: -2
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>
>> > >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> > >> java.lang.Exception: Bad command code: -2
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> > >> Caused by: java.lang.Exception: Bad command code: -2
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >> java.util.concurrent.BrokenBarrierException
>> > >> at
>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> > >> at
>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> > >> at
>> > >>
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >> at java.lang.Thread.run(Thread.java:745)
>> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception:
>> Bad
>> > >> command code: -2
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>
>> > >> java.util.concurrent.BrokenBarrierException
>> > >> at
>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> > >> at
>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> > >> at
>> > >>
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >> at java.lang.Thread.run(Thread.java:745)
>> > >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> > >> java.lang.Exception: Bad command code: -2
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> > >> Caused by: java.lang.Exception: Bad command code: -2
>> > >> at
>> > >>
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> > execution!
>> > >> java.io.IOException: Stream closed
>> > >> at
>> > >>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> > >> at
>> > >>
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >> at java.lang.Thread.run(Thread.java:745)
>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> > execution!
>> > >> java.io.IOException: Stream closed
>> > >> at
>> > >>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> > >> at
>> > >>
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >> at java.lang.Thread.run(Thread.java:745)
>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after
>> bsp
>> > >> execution.
>> > >> java.io.IOException: Stream closed
>> > >> at
>> > >>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> > >> at
>> > >>
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> > >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >> at java.lang.Thread.run(Thread.java:745)
>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
>> > execution!
>> > >> java.util.concurrent.ExecutionException: java.io.IOException: Stream
>> > closed
>> > >> at
>> java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> > >> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> > >> at java.lang.Thread.run(Thread.java:745)
>> > >> Caused by: java.io.IOException: Stream closed
>> > >> at
>> > >>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> > >> at
>> > >>
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >> ... 1 more
>> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after
>> bsp
>> > >> execution.
>> > >> java.io.IOException: Stream closed
>> > >> at
>> > >>
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >> at java.io.OutputStream.write(OutputStream.java:116)
>> > >> at
>> > >>
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >> at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >> at
>> > >>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> > >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >> at
>> > >>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >> at
>> > >>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >> at java.lang.Thread.run(Thread.java:745)
>> > >> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
>> > >>
>> > >> Vincenzo
>> > >>
>> > >> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <
>> edwardyoon@apache.org>
>> > >> wrote:
>> > >>
>> > >>> It looks like a bug of input reader. Can you try to run K-Means
>> > example?
>> > >>>
>> > >>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
>> > >>> <na...@gmail.com> wrote:
>> > >>> > Hi to all,
>> > >>> > this is my first mail to this mailing list so please have patience
>> > if I
>> > >>> > make some bad choice in the format.
>> > >>> >
>> > >>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500
>> > with 2
>> > >>> GB
>> > >>> > of RAM.
>> > >>> >
>> > >>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
>> > >>> Hama-0.7.0
>> > >>> > as you can see from the following lines:
>> > >>> >
>> > >>> >
>> > >>> >>
>> > >>> >> > $ hadoop version
>> > >>> >> > Hadoop 2.7.1
>> > >>> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git
>> -r
>> > >>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
>> > >>> >> > Compiled by jenkins on 2015-06-29T06:04Z
>> > >>> >> > Compiled with protoc 2.5.0
>> > >>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
>> > >>> >> > This command was run using
>> > >>> >
>> > >>>
>> >
>> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
>> > >>> >
>> > >>> >
>> > >>> >> > $ java -version
>> > >>> >> > java version "1.8.0_60"
>> > >>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
>> > >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>> > >>> >> >
>> > >>> >
>> > >>> > I am able to properly run the basic Hadoop and Hama examples like
>> the
>> > >>> > following:
>> > >>> > bin/hadoop jar
>> > share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
>> > >>> > grep input output 'dfs[a-z.]+'
>> > >>> > bin/hama jar hama-examples-0.7.0.jar pi
>> > >>> > My problem is that I receive the following error message when I
>> try
>> > to
>> > >>> run
>> > >>> > the HelloWorld Hama Streaming example with this instruction:
>> > >>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>> > >>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>> -program
>> > >>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> > >>> > The default python interpreters of my OS were python-2.7 and
>> > python-3.4;
>> > >>> > since I had problems with this example I also tried to install
>> > python-3.2
>> > >>> > with the following instructions but it didn't solved the problem:
>> > >>> >
>> > >>> >
>> > >>> >>
>> > >>> >> > sudo apt-get install software-properties-common
>> > >>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
>> > >>> >> > sudo apt-get update
>> > >>> >> > sudo apt-get install python3.2
>> > >>> >> >
>> > >>> >
>> > >>> >
>> > >>> > The installed python version is 3.2.6 as you can see from the
>> > following
>> > >>> > lines:
>> > >>> >
>> > >>> >
>> > >>> >>
>> > >>> >> > $ python3.2
>> > >>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
>> > >>> >> > [GCC 4.8.2] on linux2
>> > >>> >> > Type "help", "copyright", "credits" or "license" for more
>> > information.
>> > >>> >> > >>>
>> > >>> >> >
>> > >>> >
>> > >>> >
>> > >>> > The error message is the following (I am working in local mode so
>> I
>> > >>> didn't
>> > >>> > run bin/start-bspd.sh):
>> > >>> >
>> > >>> >
>> > >>> >>
>> > >>> >>
>> > >>> >> >
>> > >>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>> > >>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output
>> > /tmp/pystream-out/
>> > >>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> > >>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
>> > >>> > native-hadoop library for your platform... using builtin-java
>> classes
>> > >>> where
>> > >>> > applicable
>> > >>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation:
>> fs.default.name
>> > is
>> > >>> > deprecated. Instead, use fs.defaultFS
>> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
>> > >>> > deprecated. Instead, use mapreduce.job.user.name
>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> > >>> >
>> >
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> > >>> :an
>> > >>> > attempt to override final parameter:
>> > >>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> > >>> >
>> >
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> > >>> :an
>> > >>> > attempt to override final parameter:
>> > >>> > mapreduce.job.end-notification.max.attempts; Ignoring.
>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
>> > >>> > deprecated. Instead, use mapreduce.job.user.name
>> > >>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
>> > >>> > job_localrunner_0001
>> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
>> > >>> > mapred.cache.localFiles is deprecated. Instead, use
>> > >>> > mapreduce.job.cache.local.files
>> > >>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new
>> > barrier
>> > >>> for
>> > >>> > 2 tasks!
>> > >>> >> > java.lang.NumberFormatException: For input string: "Traceback
>> > (most
>> > >>> > recent call last):"
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> > >>> >> > java.lang.NumberFormatException: For input string: "Traceback
>> > (most
>> > >>> > recent call last):"
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>> > java.lang.Exception:
>> > >>> Bad
>> > >>> > command code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>> >> >
>> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
>> > java.lang.Exception:
>> > >>> Bad
>> > >>> > command code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>> >> >
>> > >>> >> > java.util.concurrent.BrokenBarrierException
>> > >>> >> > at
>> > >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> > >>> >> > at
>> > >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> > >>> >> > at
>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> > >>> >> > at
>> > >>> >
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> > >>> >> > Exception in thread "pipe-uplink-handler"
>> > java.lang.RuntimeException:
>> > >>> > java.lang.Exception: Bad command code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>> >> > java.util.concurrent.BrokenBarrierException
>> > >>> >> > at
>> > >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> > >>> >> > at
>> > >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> > >>> >> > at
>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> > >>> >> > at
>> > >>> >
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> > >>> >> > Exception in thread "pipe-uplink-handler"
>> > java.lang.RuntimeException:
>> > >>> > java.lang.Exception: Bad command code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>> BSP
>> > >>> > execution!
>> > >>> >> > java.io.IOException: Stream closed
>> > >>> >> > at
>> > >>> >
>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> > >>> >> > at
>> > >>> >
>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >>> >> > at
>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >>> >> > at
>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >>> >> > at
>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> > >>> >> > at
>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> > >>> >> > at
>> > >>> >
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>> BSP
>> > >>> > execution!
>> > >>> >> > java.lang.InterruptedException: java.lang.Exception: Bad
>> command
>> > code:
>> > >>> > -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>> >> >
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> > >>> >> > at
>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> > >>> >> > at
>> > >>> >
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
>> > after
>> > >>> bsp
>> > >>> > execution.
>> > >>> >> > java.io.IOException: Stream closed
>> > >>> >> > at
>> > >>> >
>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> > >>> >> > at
>> > >>> >
>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >>> >> > at
>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >>> >> > at
>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >>> >> > at
>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> > >>> >> > at
>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> > >>> >> > at
>> > >>> >
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
>> > after
>> > >>> bsp
>> > >>> > execution.
>> > >>> >> > java.io.IOException: Stream closed
>> > >>> >> > at
>> > >>> >
>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> > >>> >> > at
>> > >>> >
>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > >>> >> > at
>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > >>> >> > at
>> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > >>> >> > at
>> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> > >>> >> > at
>> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> > >>> >> > at
>> > >>> >
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during
>> BSP
>> > >>> > execution!
>> > >>> >> > java.util.concurrent.ExecutionException:
>> > >>> > java.lang.InterruptedException: java.lang.Exception: Bad command
>> > code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>> >> >
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> > >>> >> > at java.lang.Thread.run(Thread.java:745)
>> > >>> >> > Caused by: java.lang.InterruptedException: java.lang.Exception:
>> > Bad
>> > >>> > command code: -2
>> > >>> >> > at
>> > >>> >
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > >>> >> >
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> > >>> >> > at
>> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> > >>> >> > at
>> > >>> >
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > >>> >> > at
>> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > >>> >> > at
>> > >>> >
>> > >>>
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > >>> >> > ... 1 more
>> > >>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
>> > >>> >> >
>> > >>> >
>> > >>> >
>> > >>> > I tried to searching for these error messages with google but I
>> > didn't
>> > >>> find
>> > >>> > any solution.
>> > >>> >
>> > >>> > Does anyone of you have any idea of how can I solve this problem?
>> > >>> >
>> > >>> > Any help is appreciated :)
>> > >>> >
>> > >>> > Thank you
>> > >>> > Best
>> > >>> >
>> > >>> > Vincenzo
>> > >>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> Best Regards, Edward J. Yoon
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >>
>> > >>
>> >
>> -----------------------------------------------------------------------------
>> > >> Dr. Vincenzo De Leo, PhD
>> > >> Research and Development @ Linkalab
>> > >>
>> >
>> -----------------------------------------------------------------------------
>> > >> Home Page: http://www.linkalab.it/people/vincenzo.deleo
>> > >> e-mail: vincenzo.deleo@linkalab.it
>> > >> Phone: (+39)0706753140
>> > >> Mobile: (+39)3406673544
>> > >> Skype contact: vincenzo.deleo77
>> > >> GTalk contact: nagatobimaru
>> > >>
>> >
>> -----------------------------------------------------------------------------
>> > >> Linkalab (http://www.linkalab.it)
>> > >> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
>> > >> Via San Benedetto, 88
>> > >> 09129 Cagliari (CA), ITALY
>> > >> P. IVA: 03148020922
>> > >> Phone: (+39) 0707966021
>> > >> FAX: (+39) 0707962000
>> > >>
>> >
>> -----------------------------------------------------------------------------
>> > >
>> > >
>> > >
>> > > --
>> > > Best Regards, Edward J. Yoon
>> >
>> >
>> >
>> > --
>> > Best Regards, Edward J. Yoon
>> >
>>
>
>
Re: hama streaming problem with hello world example
Posted by Martin Illecker <mi...@apache.org>.
Hello Vincenzo,
I'm sorry for the inconvenience!
I'm investigating this issue and will provide a solution until tomorrow.
Best regards
Martin
2015-10-07 12:41 GMT+02:00 Vincenzo De Leo <na...@gmail.com>:
> Thank You Edward,
> can you explain to me how can I use your solution? I have tried with:
>
> - bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> -inputformat
> org.apache.hama.bsp.NullInputFormat -outputformat
> org.apache.hama.bsp.SequenceFileOutputFormat -program
> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>
> but I received the same old error messages... Do I have to install some
> patch and how can I eventually do this?
>
> Vincenzo
>
> On Mon, Oct 5, 2015 at 2:41 AM, Edward J. Yoon <ed...@apache.org>
> wrote:
>
> > Hi Vincenzo,
> >
> > I just found some code changes related to this problem at
> > https://issues.apache.org/jira/browse/HAMA-839, and I was able to run
> > Python program by adding the in/output formatters like: -inputformat
> > org.apache.hama.bsp.NullInputFormat -outputformat
> > org.apache.hama.bsp.SequenceFileOutputFormat.
> >
> > I'll check more and update the Wiki page. Sorry for inconvenient!
> >
> >
> > On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <ed...@apache.org>
> > wrote:
> > > I can look at it closely next week. BTW, should input be provided?
> > >
> > > On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
> > > <vi...@linkalab.it> wrote:
> > >> Thank you for your kind reply; this is the output of the K-Means
> > example:
> > >>
> > >> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> > python3.2
> > >> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> > >> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
> > >> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load
> > native-hadoop
> > >> library for your platform... using builtin-java classes where
> applicable
> > >> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
> > >> deprecated. Instead, use fs.defaultFS
> > >> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is
> > deprecated.
> > >> Instead, use mapreduce.job.user.name
> > >> 15/10/01 15:06:57 WARN conf.Configuration:
> > >>
> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> > :an
> > >> attempt to override final parameter:
> > >> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> > >> 15/10/01 15:06:57 WARN conf.Configuration:
> > >>
> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> > :an
> > >> attempt to override final parameter:
> > >> mapreduce.job.end-notification.max.attempts; Ignoring.
> > >> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is
> > deprecated.
> > >> Instead, use mapreduce.job.user.name
> > >> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job:
> > job_localrunner_0001
> > >> 15/10/01 15:06:57 INFO Configuration.deprecation:
> > mapred.cache.localFiles
> > >> is deprecated. Instead, use mapreduce.job.cache.local.files
> > >> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier
> for
> > 2
> > >> tasks!
> > >> java.lang.NumberFormatException: For input string: "Traceback (most
> > recent
> > >> call last):"
> > >> at
> > >>
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > >> at java.lang.Integer.parseInt(Integer.java:580)
> > >> at java.lang.Integer.parseInt(Integer.java:615)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > >> java.lang.NumberFormatException: For input string: "Traceback (most
> > recent
> > >> call last):"
> > >> at
> > >>
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > >> at java.lang.Integer.parseInt(Integer.java:580)
> > >> at java.lang.Integer.parseInt(Integer.java:615)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception:
> Bad
> > >> command code: -2
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>
> > >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> > >> java.lang.Exception: Bad command code: -2
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > >> Caused by: java.lang.Exception: Bad command code: -2
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >> java.util.concurrent.BrokenBarrierException
> > >> at
> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > >> at
> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > >> at
> > >>
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > >> at
> > >>
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception:
> Bad
> > >> command code: -2
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>
> > >> java.util.concurrent.BrokenBarrierException
> > >> at
> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > >> at
> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > >> at
> > >>
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > >> at
> > >>
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> > >> java.lang.Exception: Bad command code: -2
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > >> Caused by: java.lang.Exception: Bad command code: -2
> > >> at
> > >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> > execution!
> > >> java.io.IOException: Stream closed
> > >> at
> > >>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >> at java.io.OutputStream.write(OutputStream.java:116)
> > >> at
> > >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> > >> at
> > >>
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> > execution!
> > >> java.io.IOException: Stream closed
> > >> at
> > >>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >> at java.io.OutputStream.write(OutputStream.java:116)
> > >> at
> > >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> > >> at
> > >>
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after
> bsp
> > >> execution.
> > >> java.io.IOException: Stream closed
> > >> at
> > >>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >> at java.io.OutputStream.write(OutputStream.java:116)
> > >> at
> > >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > >> at
> > >>
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> > execution!
> > >> java.util.concurrent.ExecutionException: java.io.IOException: Stream
> > closed
> > >> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> > >> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> Caused by: java.io.IOException: Stream closed
> > >> at
> > >>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >> at java.io.OutputStream.write(OutputStream.java:116)
> > >> at
> > >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> > >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> > >> at
> > >>
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >> ... 1 more
> > >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after
> bsp
> > >> execution.
> > >> java.io.IOException: Stream closed
> > >> at
> > >>
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >> at java.io.OutputStream.write(OutputStream.java:116)
> > >> at
> > >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >> at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >> at
> > >>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > >> at
> > >>
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >> at
> > >>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >> at
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
> > >>
> > >> Vincenzo
> > >>
> > >> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <edwardyoon@apache.org
> >
> > >> wrote:
> > >>
> > >>> It looks like a bug of input reader. Can you try to run K-Means
> > example?
> > >>>
> > >>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
> > >>> <na...@gmail.com> wrote:
> > >>> > Hi to all,
> > >>> > this is my first mail to this mailing list so please have patience
> > if I
> > >>> > make some bad choice in the format.
> > >>> >
> > >>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500
> > with 2
> > >>> GB
> > >>> > of RAM.
> > >>> >
> > >>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
> > >>> Hama-0.7.0
> > >>> > as you can see from the following lines:
> > >>> >
> > >>> >
> > >>> >>
> > >>> >> > $ hadoop version
> > >>> >> > Hadoop 2.7.1
> > >>> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git
> -r
> > >>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
> > >>> >> > Compiled by jenkins on 2015-06-29T06:04Z
> > >>> >> > Compiled with protoc 2.5.0
> > >>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
> > >>> >> > This command was run using
> > >>> >
> > >>>
> >
> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
> > >>> >
> > >>> >
> > >>> >> > $ java -version
> > >>> >> > java version "1.8.0_60"
> > >>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> > >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> > >>> >> >
> > >>> >
> > >>> > I am able to properly run the basic Hadoop and Hama examples like
> the
> > >>> > following:
> > >>> > bin/hadoop jar
> > share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
> > >>> > grep input output 'dfs[a-z.]+'
> > >>> > bin/hama jar hama-examples-0.7.0.jar pi
> > >>> > My problem is that I receive the following error message when I try
> > to
> > >>> run
> > >>> > the HelloWorld Hama Streaming example with this instruction:
> > >>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> > >>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> -program
> > >>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> > >>> > The default python interpreters of my OS were python-2.7 and
> > python-3.4;
> > >>> > since I had problems with this example I also tried to install
> > python-3.2
> > >>> > with the following instructions but it didn't solved the problem:
> > >>> >
> > >>> >
> > >>> >>
> > >>> >> > sudo apt-get install software-properties-common
> > >>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
> > >>> >> > sudo apt-get update
> > >>> >> > sudo apt-get install python3.2
> > >>> >> >
> > >>> >
> > >>> >
> > >>> > The installed python version is 3.2.6 as you can see from the
> > following
> > >>> > lines:
> > >>> >
> > >>> >
> > >>> >>
> > >>> >> > $ python3.2
> > >>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
> > >>> >> > [GCC 4.8.2] on linux2
> > >>> >> > Type "help", "copyright", "credits" or "license" for more
> > information.
> > >>> >> > >>>
> > >>> >> >
> > >>> >
> > >>> >
> > >>> > The error message is the following (I am working in local mode so I
> > >>> didn't
> > >>> > run bin/start-bspd.sh):
> > >>> >
> > >>> >
> > >>> >>
> > >>> >>
> > >>> >> >
> > >>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> > >>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output
> > /tmp/pystream-out/
> > >>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> > >>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
> > >>> > native-hadoop library for your platform... using builtin-java
> classes
> > >>> where
> > >>> > applicable
> > >>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation:
> fs.default.name
> > is
> > >>> > deprecated. Instead, use fs.defaultFS
> > >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
> > >>> > deprecated. Instead, use mapreduce.job.user.name
> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> > >>> >
> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> > >>> :an
> > >>> > attempt to override final parameter:
> > >>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
> > >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> > >>> >
> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> > >>> :an
> > >>> > attempt to override final parameter:
> > >>> > mapreduce.job.end-notification.max.attempts; Ignoring.
> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
> > >>> > deprecated. Instead, use mapreduce.job.user.name
> > >>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
> > >>> > job_localrunner_0001
> > >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
> > >>> > mapred.cache.localFiles is deprecated. Instead, use
> > >>> > mapreduce.job.cache.local.files
> > >>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new
> > barrier
> > >>> for
> > >>> > 2 tasks!
> > >>> >> > java.lang.NumberFormatException: For input string: "Traceback
> > (most
> > >>> > recent call last):"
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > >>> >> > java.lang.NumberFormatException: For input string: "Traceback
> > (most
> > >>> > recent call last):"
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
> > >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
> > java.lang.Exception:
> > >>> Bad
> > >>> > command code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>> >> >
> > >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
> > java.lang.Exception:
> > >>> Bad
> > >>> > command code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>> >> >
> > >>> >> > java.util.concurrent.BrokenBarrierException
> > >>> >> > at
> > >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > >>> >> > at
> > >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > >>> >> > at
> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > >>> >> > at
> > >>> >
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> > >>> >> > Exception in thread "pipe-uplink-handler"
> > java.lang.RuntimeException:
> > >>> > java.lang.Exception: Bad command code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>> >> > java.util.concurrent.BrokenBarrierException
> > >>> >> > at
> > >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > >>> >> > at
> > >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > >>> >> > at
> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > >>> >> > at
> > >>> >
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> > >>> >> > Exception in thread "pipe-uplink-handler"
> > java.lang.RuntimeException:
> > >>> > java.lang.Exception: Bad command code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > >>> >> > Caused by: java.lang.Exception: Bad command code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> > >>> > execution!
> > >>> >> > java.io.IOException: Stream closed
> > >>> >> > at
> > >>> >
> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> > >>> >> > at
> > >>> >
> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >>> >> > at
> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >>> >> > at
> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >>> >> > at
> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> > >>> >> > at
> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> > >>> >> > at
> > >>> >
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> > >>> > execution!
> > >>> >> > java.lang.InterruptedException: java.lang.Exception: Bad command
> > code:
> > >>> > -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>> >> >
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> > >>> >> > at
> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> > >>> >> > at
> > >>> >
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
> > after
> > >>> bsp
> > >>> > execution.
> > >>> >> > java.io.IOException: Stream closed
> > >>> >> > at
> > >>> >
> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> > >>> >> > at
> > >>> >
> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >>> >> > at
> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >>> >> > at
> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >>> >> > at
> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > >>> >> > at
> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > >>> >> > at
> > >>> >
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
> > after
> > >>> bsp
> > >>> > execution.
> > >>> >> > java.io.IOException: Stream closed
> > >>> >> > at
> > >>> >
> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> > >>> >> > at
> > >>> >
> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >>> >> > at
> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >>> >> > at
> > >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >>> >> > at
> > java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > >>> >> > at
> > org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > >>> >> > at
> > >>> >
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> > >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> > >>> > execution!
> > >>> >> > java.util.concurrent.ExecutionException:
> > >>> > java.lang.InterruptedException: java.lang.Exception: Bad command
> > code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>> >> >
> > >>> >> > at
> > java.util.concurrent.FutureTask.report(FutureTask.java:122)
> > >>> >> > at
> > java.util.concurrent.FutureTask.get(FutureTask.java:192)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> > >>> >> > at java.lang.Thread.run(Thread.java:745)
> > >>> >> > Caused by: java.lang.InterruptedException: java.lang.Exception:
> > Bad
> > >>> > command code: -2
> > >>> >> > at
> > >>> >
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >>> >> >
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> > >>> >> > at
> org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> > >>> >> > at
> > >>> >
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >>> >> > at
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> >> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >>> >> > at
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >>> >> > ... 1 more
> > >>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
> > >>> >> >
> > >>> >
> > >>> >
> > >>> > I tried to searching for these error messages with google but I
> > didn't
> > >>> find
> > >>> > any solution.
> > >>> >
> > >>> > Does anyone of you have any idea of how can I solve this problem?
> > >>> >
> > >>> > Any help is appreciated :)
> > >>> >
> > >>> > Thank you
> > >>> > Best
> > >>> >
> > >>> > Vincenzo
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Best Regards, Edward J. Yoon
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >>
> > >>
> >
> -----------------------------------------------------------------------------
> > >> Dr. Vincenzo De Leo, PhD
> > >> Research and Development @ Linkalab
> > >>
> >
> -----------------------------------------------------------------------------
> > >> Home Page: http://www.linkalab.it/people/vincenzo.deleo
> > >> e-mail: vincenzo.deleo@linkalab.it
> > >> Phone: (+39)0706753140
> > >> Mobile: (+39)3406673544
> > >> Skype contact: vincenzo.deleo77
> > >> GTalk contact: nagatobimaru
> > >>
> >
> -----------------------------------------------------------------------------
> > >> Linkalab (http://www.linkalab.it)
> > >> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
> > >> Via San Benedetto, 88
> > >> 09129 Cagliari (CA), ITALY
> > >> P. IVA: 03148020922
> > >> Phone: (+39) 0707966021
> > >> FAX: (+39) 0707962000
> > >>
> >
> -----------------------------------------------------------------------------
> > >
> > >
> > >
> > > --
> > > Best Regards, Edward J. Yoon
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon
> >
>
Re: hama streaming problem with hello world example
Posted by Vincenzo De Leo <na...@gmail.com>.
Thank You Edward,
can you explain to me how can I use your solution? I have tried with:
- bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
-cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -inputformat
org.apache.hama.bsp.NullInputFormat -outputformat
org.apache.hama.bsp.SequenceFileOutputFormat -program
/tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
but I received the same old error messages... Do I have to install some
patch and how can I eventually do this?
Vincenzo
On Mon, Oct 5, 2015 at 2:41 AM, Edward J. Yoon <ed...@apache.org>
wrote:
> Hi Vincenzo,
>
> I just found some code changes related to this problem at
> https://issues.apache.org/jira/browse/HAMA-839, and I was able to run
> Python program by adding the in/output formatters like: -inputformat
> org.apache.hama.bsp.NullInputFormat -outputformat
> org.apache.hama.bsp.SequenceFileOutputFormat.
>
> I'll check more and update the Wiki page. Sorry for inconvenient!
>
>
> On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <ed...@apache.org>
> wrote:
> > I can look at it closely next week. BTW, should input be provided?
> >
> > On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
> > <vi...@linkalab.it> wrote:
> >> Thank you for your kind reply; this is the output of the K-Means
> example:
> >>
> >> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> python3.2
> >> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> >> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
> >> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load
> native-hadoop
> >> library for your platform... using builtin-java classes where applicable
> >> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
> >> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
> >> deprecated. Instead, use fs.defaultFS
> >> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is
> deprecated.
> >> Instead, use mapreduce.job.user.name
> >> 15/10/01 15:06:57 WARN conf.Configuration:
> >>
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> :an
> >> attempt to override final parameter:
> >> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> >> 15/10/01 15:06:57 WARN conf.Configuration:
> >>
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> :an
> >> attempt to override final parameter:
> >> mapreduce.job.end-notification.max.attempts; Ignoring.
> >> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is
> deprecated.
> >> Instead, use mapreduce.job.user.name
> >> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job:
> job_localrunner_0001
> >> 15/10/01 15:06:57 INFO Configuration.deprecation:
> mapred.cache.localFiles
> >> is deprecated. Instead, use mapreduce.job.cache.local.files
> >> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier for
> 2
> >> tasks!
> >> java.lang.NumberFormatException: For input string: "Traceback (most
> recent
> >> call last):"
> >> at
> >>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >> at java.lang.Integer.parseInt(Integer.java:580)
> >> at java.lang.Integer.parseInt(Integer.java:615)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >> java.lang.NumberFormatException: For input string: "Traceback (most
> recent
> >> call last):"
> >> at
> >>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >> at java.lang.Integer.parseInt(Integer.java:580)
> >> at java.lang.Integer.parseInt(Integer.java:615)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> >> command code: -2
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>
> >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> >> java.lang.Exception: Bad command code: -2
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >> Caused by: java.lang.Exception: Bad command code: -2
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> java.util.concurrent.BrokenBarrierException
> >> at
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >> at
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >> at
> >> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> >> command code: -2
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>
> >> java.util.concurrent.BrokenBarrierException
> >> at
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >> at
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >> at
> >> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> >> java.lang.Exception: Bad command code: -2
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >> Caused by: java.lang.Exception: Bad command code: -2
> >> at
> >> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
> >> java.io.IOException: Stream closed
> >> at
> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> at java.io.OutputStream.write(OutputStream.java:116)
> >> at
> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
> >> java.io.IOException: Stream closed
> >> at
> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> at java.io.OutputStream.write(OutputStream.java:116)
> >> at
> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> >> execution.
> >> java.io.IOException: Stream closed
> >> at
> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> at java.io.OutputStream.write(OutputStream.java:116)
> >> at
> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
> >> java.util.concurrent.ExecutionException: java.io.IOException: Stream
> closed
> >> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: java.io.IOException: Stream closed
> >> at
> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> at java.io.OutputStream.write(OutputStream.java:116)
> >> at
> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> ... 1 more
> >> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> >> execution.
> >> java.io.IOException: Stream closed
> >> at
> >> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> at java.io.OutputStream.write(OutputStream.java:116)
> >> at
> >> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> at
> >>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> at
> >>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
> >>
> >> Vincenzo
> >>
> >> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <ed...@apache.org>
> >> wrote:
> >>
> >>> It looks like a bug of input reader. Can you try to run K-Means
> example?
> >>>
> >>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
> >>> <na...@gmail.com> wrote:
> >>> > Hi to all,
> >>> > this is my first mail to this mailing list so please have patience
> if I
> >>> > make some bad choice in the format.
> >>> >
> >>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500
> with 2
> >>> GB
> >>> > of RAM.
> >>> >
> >>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
> >>> Hama-0.7.0
> >>> > as you can see from the following lines:
> >>> >
> >>> >
> >>> >>
> >>> >> > $ hadoop version
> >>> >> > Hadoop 2.7.1
> >>> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
> >>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
> >>> >> > Compiled by jenkins on 2015-06-29T06:04Z
> >>> >> > Compiled with protoc 2.5.0
> >>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
> >>> >> > This command was run using
> >>> >
> >>>
> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
> >>> >
> >>> >
> >>> >> > $ java -version
> >>> >> > java version "1.8.0_60"
> >>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> >>> >> >
> >>> >
> >>> > I am able to properly run the basic Hadoop and Hama examples like the
> >>> > following:
> >>> > bin/hadoop jar
> share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
> >>> > grep input output 'dfs[a-z.]+'
> >>> > bin/hama jar hama-examples-0.7.0.jar pi
> >>> > My problem is that I receive the following error message when I try
> to
> >>> run
> >>> > the HelloWorld Hama Streaming example with this instruction:
> >>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> >>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> >>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> >>> > The default python interpreters of my OS were python-2.7 and
> python-3.4;
> >>> > since I had problems with this example I also tried to install
> python-3.2
> >>> > with the following instructions but it didn't solved the problem:
> >>> >
> >>> >
> >>> >>
> >>> >> > sudo apt-get install software-properties-common
> >>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
> >>> >> > sudo apt-get update
> >>> >> > sudo apt-get install python3.2
> >>> >> >
> >>> >
> >>> >
> >>> > The installed python version is 3.2.6 as you can see from the
> following
> >>> > lines:
> >>> >
> >>> >
> >>> >>
> >>> >> > $ python3.2
> >>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
> >>> >> > [GCC 4.8.2] on linux2
> >>> >> > Type "help", "copyright", "credits" or "license" for more
> information.
> >>> >> > >>>
> >>> >> >
> >>> >
> >>> >
> >>> > The error message is the following (I am working in local mode so I
> >>> didn't
> >>> > run bin/start-bspd.sh):
> >>> >
> >>> >
> >>> >>
> >>> >>
> >>> >> >
> >>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> >>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output
> /tmp/pystream-out/
> >>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> >>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
> >>> > native-hadoop library for your platform... using builtin-java classes
> >>> where
> >>> > applicable
> >>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
> >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name
> is
> >>> > deprecated. Instead, use fs.defaultFS
> >>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
> >>> > deprecated. Instead, use mapreduce.job.user.name
> >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> >>> >
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> >>> :an
> >>> > attempt to override final parameter:
> >>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
> >>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> >>> >
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> >>> :an
> >>> > attempt to override final parameter:
> >>> > mapreduce.job.end-notification.max.attempts; Ignoring.
> >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
> >>> > deprecated. Instead, use mapreduce.job.user.name
> >>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
> >>> > job_localrunner_0001
> >>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
> >>> > mapred.cache.localFiles is deprecated. Instead, use
> >>> > mapreduce.job.cache.local.files
> >>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new
> barrier
> >>> for
> >>> > 2 tasks!
> >>> >> > java.lang.NumberFormatException: For input string: "Traceback
> (most
> >>> > recent call last):"
> >>> >> > at
> >>> >
> >>>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
> >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >>> >> > java.lang.NumberFormatException: For input string: "Traceback
> (most
> >>> > recent call last):"
> >>> >> > at
> >>> >
> >>>
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >>> >> > at java.lang.Integer.parseInt(Integer.java:580)
> >>> >> > at java.lang.Integer.parseInt(Integer.java:615)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
> java.lang.Exception:
> >>> Bad
> >>> > command code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> >> >
> >>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader:
> java.lang.Exception:
> >>> Bad
> >>> > command code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> >> >
> >>> >> > java.util.concurrent.BrokenBarrierException
> >>> >> > at
> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >>> >> > at
> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >>> >> > at
> >>> >
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >>> >> > at
> >>> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> >> > Exception in thread "pipe-uplink-handler"
> java.lang.RuntimeException:
> >>> > java.lang.Exception: Bad command code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >>> >> > Caused by: java.lang.Exception: Bad command code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> >> > java.util.concurrent.BrokenBarrierException
> >>> >> > at
> >>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >>> >> > at
> >>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >>> >> > at
> >>> >
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >>> >> > at
> >>> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> >> > Exception in thread "pipe-uplink-handler"
> java.lang.RuntimeException:
> >>> > java.lang.Exception: Bad command code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >>> >> > Caused by: java.lang.Exception: Bad command code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> >>> > execution!
> >>> >> > java.io.IOException: Stream closed
> >>> >> > at
> >>> >
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >>> >> > at
> >>> >
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> >> > at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> >> > at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> >> > at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >>> >> > at
> >>> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> >>> > execution!
> >>> >> > java.lang.InterruptedException: java.lang.Exception: Bad command
> code:
> >>> > -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> >> >
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> >>> >> > at
> >>> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
> after
> >>> bsp
> >>> > execution.
> >>> >> > java.io.IOException: Stream closed
> >>> >> > at
> >>> >
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >>> >> > at
> >>> >
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> >> > at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> >> > at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> >> > at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >>> >> > at
> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >>> >> > at
> >>> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up
> after
> >>> bsp
> >>> > execution.
> >>> >> > java.io.IOException: Stream closed
> >>> >> > at
> >>> >
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >>> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >>> >> > at
> >>> >
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >>> >> > at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >>> >> > at
> >>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >>> >> > at
> java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >>> >> > at
> org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >>> >> > at
> >>> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> >>> > execution!
> >>> >> > java.util.concurrent.ExecutionException:
> >>> > java.lang.InterruptedException: java.lang.Exception: Bad command
> code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> >> >
> >>> >> > at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >>> >> > at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> >>> >> > at java.lang.Thread.run(Thread.java:745)
> >>> >> > Caused by: java.lang.InterruptedException: java.lang.Exception:
> Bad
> >>> > command code: -2
> >>> >> > at
> >>> >
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >>> >> >
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> >>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> >>> >> > at
> >>> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>> >> > at
> >>> >
> >>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> >> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> >> > at
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>> >> > ... 1 more
> >>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
> >>> >> >
> >>> >
> >>> >
> >>> > I tried to searching for these error messages with google but I
> didn't
> >>> find
> >>> > any solution.
> >>> >
> >>> > Does anyone of you have any idea of how can I solve this problem?
> >>> >
> >>> > Any help is appreciated :)
> >>> >
> >>> > Thank you
> >>> > Best
> >>> >
> >>> > Vincenzo
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards, Edward J. Yoon
> >>>
> >>
> >>
> >>
> >> --
> >>
> >>
> -----------------------------------------------------------------------------
> >> Dr. Vincenzo De Leo, PhD
> >> Research and Development @ Linkalab
> >>
> -----------------------------------------------------------------------------
> >> Home Page: http://www.linkalab.it/people/vincenzo.deleo
> >> e-mail: vincenzo.deleo@linkalab.it
> >> Phone: (+39)0706753140
> >> Mobile: (+39)3406673544
> >> Skype contact: vincenzo.deleo77
> >> GTalk contact: nagatobimaru
> >>
> -----------------------------------------------------------------------------
> >> Linkalab (http://www.linkalab.it)
> >> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
> >> Via San Benedetto, 88
> >> 09129 Cagliari (CA), ITALY
> >> P. IVA: 03148020922
> >> Phone: (+39) 0707966021
> >> FAX: (+39) 0707962000
> >>
> -----------------------------------------------------------------------------
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon
>
>
>
> --
> Best Regards, Edward J. Yoon
>
Re: hama streaming problem with hello world example
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hi Vincenzo,
I just found some code changes related to this problem at
https://issues.apache.org/jira/browse/HAMA-839, and I was able to run
Python program by adding the in/output formatters like: -inputformat
org.apache.hama.bsp.NullInputFormat -outputformat
org.apache.hama.bsp.SequenceFileOutputFormat.
I'll check more and update the Wiki page. Sorry for inconvenient!
On Fri, Oct 2, 2015 at 1:29 PM, Edward J. Yoon <ed...@apache.org> wrote:
> I can look at it closely next week. BTW, should input be provided?
>
> On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
> <vi...@linkalab.it> wrote:
>> Thank you for your kind reply; this is the output of the K-Means example:
>>
>> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
>> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
>> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
>> deprecated. Instead, use fs.defaultFS
>> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is deprecated.
>> Instead, use mapreduce.job.user.name
>> 15/10/01 15:06:57 WARN conf.Configuration:
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
>> attempt to override final parameter:
>> mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> 15/10/01 15:06:57 WARN conf.Configuration:
>> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
>> attempt to override final parameter:
>> mapreduce.job.end-notification.max.attempts; Ignoring.
>> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is deprecated.
>> Instead, use mapreduce.job.user.name
>> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
>> 15/10/01 15:06:57 INFO Configuration.deprecation: mapred.cache.localFiles
>> is deprecated. Instead, use mapreduce.job.cache.local.files
>> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier for 2
>> tasks!
>> java.lang.NumberFormatException: For input string: "Traceback (most recent
>> call last):"
>> at
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> at java.lang.Integer.parseInt(Integer.java:580)
>> at java.lang.Integer.parseInt(Integer.java:615)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> java.lang.NumberFormatException: For input string: "Traceback (most recent
>> call last):"
>> at
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> at java.lang.Integer.parseInt(Integer.java:580)
>> at java.lang.Integer.parseInt(Integer.java:615)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
>> command code: -2
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>
>> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> java.lang.Exception: Bad command code: -2
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> Caused by: java.lang.Exception: Bad command code: -2
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> java.util.concurrent.BrokenBarrierException
>> at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> at
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
>> command code: -2
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>
>> java.util.concurrent.BrokenBarrierException
>> at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> at
>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> java.lang.Exception: Bad command code: -2
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> Caused by: java.lang.Exception: Bad command code: -2
>> at
>> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
>> java.io.IOException: Stream closed
>> at
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> at java.io.OutputStream.write(OutputStream.java:116)
>> at
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
>> java.io.IOException: Stream closed
>> at
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> at java.io.OutputStream.write(OutputStream.java:116)
>> at
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
>> execution.
>> java.io.IOException: Stream closed
>> at
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> at java.io.OutputStream.write(OutputStream.java:116)
>> at
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
>> java.util.concurrent.ExecutionException: java.io.IOException: Stream closed
>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.io.IOException: Stream closed
>> at
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> at java.io.OutputStream.write(OutputStream.java:116)
>> at
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> ... 1 more
>> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
>> execution.
>> java.io.IOException: Stream closed
>> at
>> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> at java.io.OutputStream.write(OutputStream.java:116)
>> at
>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> at
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> at
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
>>
>> Vincenzo
>>
>> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>>
>>> It looks like a bug of input reader. Can you try to run K-Means example?
>>>
>>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
>>> <na...@gmail.com> wrote:
>>> > Hi to all,
>>> > this is my first mail to this mailing list so please have patience if I
>>> > make some bad choice in the format.
>>> >
>>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500 with 2
>>> GB
>>> > of RAM.
>>> >
>>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
>>> Hama-0.7.0
>>> > as you can see from the following lines:
>>> >
>>> >
>>> >>
>>> >> > $ hadoop version
>>> >> > Hadoop 2.7.1
>>> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
>>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
>>> >> > Compiled by jenkins on 2015-06-29T06:04Z
>>> >> > Compiled with protoc 2.5.0
>>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
>>> >> > This command was run using
>>> >
>>> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
>>> >
>>> >
>>> >> > $ java -version
>>> >> > java version "1.8.0_60"
>>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
>>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>>> >> >
>>> >
>>> > I am able to properly run the basic Hadoop and Hama examples like the
>>> > following:
>>> > bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
>>> > grep input output 'dfs[a-z.]+'
>>> > bin/hama jar hama-examples-0.7.0.jar pi
>>> > My problem is that I receive the following error message when I try to
>>> run
>>> > the HelloWorld Hama Streaming example with this instruction:
>>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>>> > The default python interpreters of my OS were python-2.7 and python-3.4;
>>> > since I had problems with this example I also tried to install python-3.2
>>> > with the following instructions but it didn't solved the problem:
>>> >
>>> >
>>> >>
>>> >> > sudo apt-get install software-properties-common
>>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
>>> >> > sudo apt-get update
>>> >> > sudo apt-get install python3.2
>>> >> >
>>> >
>>> >
>>> > The installed python version is 3.2.6 as you can see from the following
>>> > lines:
>>> >
>>> >
>>> >>
>>> >> > $ python3.2
>>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
>>> >> > [GCC 4.8.2] on linux2
>>> >> > Type "help", "copyright", "credits" or "license" for more information.
>>> >> > >>>
>>> >> >
>>> >
>>> >
>>> > The error message is the following (I am working in local mode so I
>>> didn't
>>> > run bin/start-bspd.sh):
>>> >
>>> >
>>> >>
>>> >>
>>> >> >
>>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
>>> > native-hadoop library for your platform... using builtin-java classes
>>> where
>>> > applicable
>>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
>>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name is
>>> > deprecated. Instead, use fs.defaultFS
>>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
>>> > deprecated. Instead, use mapreduce.job.user.name
>>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>>> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>>> :an
>>> > attempt to override final parameter:
>>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
>>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>>> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>>> :an
>>> > attempt to override final parameter:
>>> > mapreduce.job.end-notification.max.attempts; Ignoring.
>>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
>>> > deprecated. Instead, use mapreduce.job.user.name
>>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
>>> > job_localrunner_0001
>>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
>>> > mapred.cache.localFiles is deprecated. Instead, use
>>> > mapreduce.job.cache.local.files
>>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new barrier
>>> for
>>> > 2 tasks!
>>> >> > java.lang.NumberFormatException: For input string: "Traceback (most
>>> > recent call last):"
>>> >> > at
>>> >
>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>>> >> > java.lang.NumberFormatException: For input string: "Traceback (most
>>> > recent call last):"
>>> >> > at
>>> >
>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception:
>>> Bad
>>> > command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> >> >
>>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception:
>>> Bad
>>> > command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> >> >
>>> >> > java.util.concurrent.BrokenBarrierException
>>> >> > at
>>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>>> >> > at
>>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>>> >> > at
>>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>>> >> > at
>>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> >> > at java.lang.Thread.run(Thread.java:745)
>>> >> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>>> > java.lang.Exception: Bad command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>>> >> > Caused by: java.lang.Exception: Bad command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> >> > java.util.concurrent.BrokenBarrierException
>>> >> > at
>>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>>> >> > at
>>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>>> >> > at
>>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>>> >> > at
>>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> >> > at java.lang.Thread.run(Thread.java:745)
>>> >> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>>> > java.lang.Exception: Bad command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>>> >> > Caused by: java.lang.Exception: Bad command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> > execution!
>>> >> > java.io.IOException: Stream closed
>>> >> > at
>>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>>> >> > at
>>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> >> > at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> >> > at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>>> >> > at
>>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> >> > at java.lang.Thread.run(Thread.java:745)
>>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> > execution!
>>> >> > java.lang.InterruptedException: java.lang.Exception: Bad command code:
>>> > -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> >> >
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>>> >> > at
>>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> >> > at java.lang.Thread.run(Thread.java:745)
>>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after
>>> bsp
>>> > execution.
>>> >> > java.io.IOException: Stream closed
>>> >> > at
>>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>>> >> > at
>>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> >> > at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> >> > at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>>> >> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>>> >> > at
>>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> >> > at java.lang.Thread.run(Thread.java:745)
>>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after
>>> bsp
>>> > execution.
>>> >> > java.io.IOException: Stream closed
>>> >> > at
>>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>>> >> > at
>>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>> >> > at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>>> >> > at
>>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>>> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>>> >> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>>> >> > at
>>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> >> > at java.lang.Thread.run(Thread.java:745)
>>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> > execution!
>>> >> > java.util.concurrent.ExecutionException:
>>> > java.lang.InterruptedException: java.lang.Exception: Bad command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> >> >
>>> >> > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>>> >> > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>>> >> > at java.lang.Thread.run(Thread.java:745)
>>> >> > Caused by: java.lang.InterruptedException: java.lang.Exception: Bad
>>> > command code: -2
>>> >> > at
>>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>>> >> >
>>> >> > at
>>> >
>>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>>> >> > at
>>> >
>>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>>> >> > at
>>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>> >> > at
>>> >
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> >> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> >> > ... 1 more
>>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
>>> >> >
>>> >
>>> >
>>> > I tried to searching for these error messages with google but I didn't
>>> find
>>> > any solution.
>>> >
>>> > Does anyone of you have any idea of how can I solve this problem?
>>> >
>>> > Any help is appreciated :)
>>> >
>>> > Thank you
>>> > Best
>>> >
>>> > Vincenzo
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>>
>>
>>
>>
>> --
>>
>> -----------------------------------------------------------------------------
>> Dr. Vincenzo De Leo, PhD
>> Research and Development @ Linkalab
>> -----------------------------------------------------------------------------
>> Home Page: http://www.linkalab.it/people/vincenzo.deleo
>> e-mail: vincenzo.deleo@linkalab.it
>> Phone: (+39)0706753140
>> Mobile: (+39)3406673544
>> Skype contact: vincenzo.deleo77
>> GTalk contact: nagatobimaru
>> -----------------------------------------------------------------------------
>> Linkalab (http://www.linkalab.it)
>> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
>> Via San Benedetto, 88
>> 09129 Cagliari (CA), ITALY
>> P. IVA: 03148020922
>> Phone: (+39) 0707966021
>> FAX: (+39) 0707962000
>> -----------------------------------------------------------------------------
>
>
>
> --
> Best Regards, Edward J. Yoon
--
Best Regards, Edward J. Yoon
Re: hama streaming problem with hello world example
Posted by "Edward J. Yoon" <ed...@apache.org>.
I can look at it closely next week. BTW, should input be provided?
On Thu, Oct 1, 2015 at 10:13 PM, Vincenzo De Leo
<vi...@linkalab.it> wrote:
> Thank you for your kind reply; this is the output of the K-Means example:
>
> $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> /tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
> 15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
> 15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 15/10/01 15:06:56 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 15/10/01 15:06:57 WARN conf.Configuration:
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/10/01 15:06:57 WARN conf.Configuration:
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/10/01 15:06:57 INFO Configuration.deprecation: user.name is deprecated.
> Instead, use mapreduce.job.user.name
> 15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
> 15/10/01 15:06:57 INFO Configuration.deprecation: mapred.cache.localFiles
> is deprecated. Instead, use mapreduce.job.cache.local.files
> 15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier for 2
> tasks!
> java.lang.NumberFormatException: For input string: "Traceback (most recent
> call last):"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580)
> at java.lang.Integer.parseInt(Integer.java:615)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> java.lang.NumberFormatException: For input string: "Traceback (most recent
> call last):"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:580)
> at java.lang.Integer.parseInt(Integer.java:615)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> command code: -2
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>
> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> java.lang.Exception: Bad command code: -2
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> Caused by: java.lang.Exception: Bad command code: -2
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> java.util.concurrent.BrokenBarrierException
> at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> at
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> command code: -2
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>
> java.util.concurrent.BrokenBarrierException
> at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> at
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> java.lang.Exception: Bad command code: -2
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> Caused by: java.lang.Exception: Bad command code: -2
> at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
> java.io.IOException: Stream closed
> at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> at java.io.OutputStream.write(OutputStream.java:116)
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
> java.io.IOException: Stream closed
> at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> at java.io.OutputStream.write(OutputStream.java:116)
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> execution.
> java.io.IOException: Stream closed
> at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> at java.io.OutputStream.write(OutputStream.java:116)
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
> java.util.concurrent.ExecutionException: java.io.IOException: Stream closed
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Stream closed
> at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> at java.io.OutputStream.write(OutputStream.java:116)
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ... 1 more
> 15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> execution.
> java.io.IOException: Stream closed
> at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> at java.io.OutputStream.write(OutputStream.java:116)
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> at
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
>
> Vincenzo
>
> On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <ed...@apache.org>
> wrote:
>
>> It looks like a bug of input reader. Can you try to run K-Means example?
>>
>> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
>> <na...@gmail.com> wrote:
>> > Hi to all,
>> > this is my first mail to this mailing list so please have patience if I
>> > make some bad choice in the format.
>> >
>> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500 with 2
>> GB
>> > of RAM.
>> >
>> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
>> Hama-0.7.0
>> > as you can see from the following lines:
>> >
>> >
>> >>
>> >> > $ hadoop version
>> >> > Hadoop 2.7.1
>> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
>> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
>> >> > Compiled by jenkins on 2015-06-29T06:04Z
>> >> > Compiled with protoc 2.5.0
>> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
>> >> > This command was run using
>> >
>> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
>> >
>> >
>> >> > $ java -version
>> >> > java version "1.8.0_60"
>> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
>> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>> >> >
>> >
>> > I am able to properly run the basic Hadoop and Hama examples like the
>> > following:
>> > bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
>> > grep input output 'dfs[a-z.]+'
>> > bin/hama jar hama-examples-0.7.0.jar pi
>> > My problem is that I receive the following error message when I try to
>> run
>> > the HelloWorld Hama Streaming example with this instruction:
>> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> > The default python interpreters of my OS were python-2.7 and python-3.4;
>> > since I had problems with this example I also tried to install python-3.2
>> > with the following instructions but it didn't solved the problem:
>> >
>> >
>> >>
>> >> > sudo apt-get install software-properties-common
>> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
>> >> > sudo apt-get update
>> >> > sudo apt-get install python3.2
>> >> >
>> >
>> >
>> > The installed python version is 3.2.6 as you can see from the following
>> > lines:
>> >
>> >
>> >>
>> >> > $ python3.2
>> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
>> >> > [GCC 4.8.2] on linux2
>> >> > Type "help", "copyright", "credits" or "license" for more information.
>> >> > >>>
>> >> >
>> >
>> >
>> > The error message is the following (I am working in local mode so I
>> didn't
>> > run bin/start-bspd.sh):
>> >
>> >
>> >>
>> >>
>> >> >
>> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
>> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
>> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
>> > native-hadoop library for your platform... using builtin-java classes
>> where
>> > applicable
>> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name is
>> > deprecated. Instead, use fs.defaultFS
>> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
>> > deprecated. Instead, use mapreduce.job.user.name
>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> :an
>> > attempt to override final parameter:
>> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> >> > 15/09/30 12:39:11 WARN conf.Configuration:
>> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
>> :an
>> > attempt to override final parameter:
>> > mapreduce.job.end-notification.max.attempts; Ignoring.
>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
>> > deprecated. Instead, use mapreduce.job.user.name
>> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
>> > job_localrunner_0001
>> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
>> > mapred.cache.localFiles is deprecated. Instead, use
>> > mapreduce.job.cache.local.files
>> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new barrier
>> for
>> > 2 tasks!
>> >> > java.lang.NumberFormatException: For input string: "Traceback (most
>> > recent call last):"
>> >> > at
>> >
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >> > java.lang.NumberFormatException: For input string: "Traceback (most
>> > recent call last):"
>> >> > at
>> >
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> >> > at java.lang.Integer.parseInt(Integer.java:580)
>> >> > at java.lang.Integer.parseInt(Integer.java:615)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception:
>> Bad
>> > command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> >
>> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception:
>> Bad
>> > command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> >
>> >> > java.util.concurrent.BrokenBarrierException
>> >> > at
>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >> > at
>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >> > at
>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >> > at
>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> > java.lang.Exception: Bad command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> > java.util.concurrent.BrokenBarrierException
>> >> > at
>> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> >> > at
>> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> >> > at
>> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> >> > at
>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
>> > java.lang.Exception: Bad command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> >> > Caused by: java.lang.Exception: Bad command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>> > execution!
>> >> > java.io.IOException: Stream closed
>> >> > at
>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >> > at
>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> > at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> > at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> >> > at
>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>> > execution!
>> >> > java.lang.InterruptedException: java.lang.Exception: Bad command code:
>> > -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> >
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> >> > at
>> >
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> >> > at
>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after
>> bsp
>> > execution.
>> >> > java.io.IOException: Stream closed
>> >> > at
>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >> > at
>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> > at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> > at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >> > at
>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after
>> bsp
>> > execution.
>> >> > java.io.IOException: Stream closed
>> >> > at
>> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> >> > at java.io.OutputStream.write(OutputStream.java:116)
>> >> > at
>> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> >> > at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> >> > at
>> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> >> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> >> > at
>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
>> > execution!
>> >> > java.util.concurrent.ExecutionException:
>> > java.lang.InterruptedException: java.lang.Exception: Bad command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> >
>> >> > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> >> > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > Caused by: java.lang.InterruptedException: java.lang.Exception: Bad
>> > command code: -2
>> >> > at
>> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >> >
>> >> > at
>> >
>> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> >> > at
>> >
>> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> >> > at
>> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> >> > at
>> >
>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> >> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> >> > ... 1 more
>> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
>> >> >
>> >
>> >
>> > I tried to searching for these error messages with google but I didn't
>> find
>> > any solution.
>> >
>> > Does anyone of you have any idea of how can I solve this problem?
>> >
>> > Any help is appreciated :)
>> >
>> > Thank you
>> > Best
>> >
>> > Vincenzo
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>>
>
>
>
> --
>
> -----------------------------------------------------------------------------
> Dr. Vincenzo De Leo, PhD
> Research and Development @ Linkalab
> -----------------------------------------------------------------------------
> Home Page: http://www.linkalab.it/people/vincenzo.deleo
> e-mail: vincenzo.deleo@linkalab.it
> Phone: (+39)0706753140
> Mobile: (+39)3406673544
> Skype contact: vincenzo.deleo77
> GTalk contact: nagatobimaru
> -----------------------------------------------------------------------------
> Linkalab (http://www.linkalab.it)
> CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
> Via San Benedetto, 88
> 09129 Cagliari (CA), ITALY
> P. IVA: 03148020922
> Phone: (+39) 0707966021
> FAX: (+39) 0707962000
> -----------------------------------------------------------------------------
--
Best Regards, Edward J. Yoon
Re: hama streaming problem with hello world example
Posted by Vincenzo De Leo <vi...@linkalab.it>.
Thank you for your kind reply; this is the output of the K-Means example:
$ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
-cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
/tmp/PyStreaming/BSPRunner.py -programArgs KMeansBSP
15/10/01 15:06:56 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
15/10/01 15:06:56 INFO pipes.Submitter: Streaming enabled!
15/10/01 15:06:56 INFO Configuration.deprecation: fs.default.name is
deprecated. Instead, use fs.defaultFS
15/10/01 15:06:56 INFO Configuration.deprecation: user.name is deprecated.
Instead, use mapreduce.job.user.name
15/10/01 15:06:57 WARN conf.Configuration:
org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/10/01 15:06:57 WARN conf.Configuration:
org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/10/01 15:06:57 INFO Configuration.deprecation: user.name is deprecated.
Instead, use mapreduce.job.user.name
15/10/01 15:06:57 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
15/10/01 15:06:57 INFO Configuration.deprecation: mapred.cache.localFiles
is deprecated. Instead, use mapreduce.job.cache.local.files
15/10/01 15:06:57 INFO bsp.LocalBSPRunner: Setting up a new barrier for 2
tasks!
java.lang.NumberFormatException: For input string: "Traceback (most recent
call last):"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at
org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
java.lang.NumberFormatException: For input string: "Traceback (most recent
call last):"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at
org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
command code: -2
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
java.lang.Exception: Bad command code: -2
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
Caused by: java.lang.Exception: Bad command code: -2
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
java.util.concurrent.BrokenBarrierException
at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
at
org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
at
org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
15/10/01 15:06:57 ERROR protocol.UplinkReader: java.lang.Exception: Bad
command code: -2
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
java.util.concurrent.BrokenBarrierException
at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
at
org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
at
org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
java.lang.Exception: Bad command code: -2
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
Caused by: java.lang.Exception: Bad command code: -2
at
org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.io.IOException: Stream closed
at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
at java.io.OutputStream.write(OutputStream.java:116)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
at
org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.io.IOException: Stream closed
at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
at java.io.OutputStream.write(OutputStream.java:116)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
at
org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
execution.
java.io.IOException: Stream closed
at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
at java.io.OutputStream.write(OutputStream.java:116)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
at
org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.util.concurrent.ExecutionException: java.io.IOException: Stream closed
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at
org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Stream closed
at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
at java.io.OutputStream.write(OutputStream.java:116)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
at
org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
15/10/01 15:06:57 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
execution.
java.io.IOException: Stream closed
at
java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
at java.io.OutputStream.write(OutputStream.java:116)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
at
org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
15/10/01 15:06:58 INFO bsp.BSPJobClient: Job failed.
Vincenzo
On Thu, Oct 1, 2015 at 1:10 AM, Edward J. Yoon <ed...@apache.org>
wrote:
> It looks like a bug of input reader. Can you try to run K-Means example?
>
> On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
> <na...@gmail.com> wrote:
> > Hi to all,
> > this is my first mail to this mailing list so please have patience if I
> > make some bad choice in the format.
> >
> > I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500 with 2
> GB
> > of RAM.
> >
> > I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e
> Hama-0.7.0
> > as you can see from the following lines:
> >
> >
> >>
> >> > $ hadoop version
> >> > Hadoop 2.7.1
> >> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
> > 15ecc87ccf4a0228f35af08fc56de536e6ce657a
> >> > Compiled by jenkins on 2015-06-29T06:04Z
> >> > Compiled with protoc 2.5.0
> >> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
> >> > This command was run using
> >
> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
> >
> >
> >> > $ java -version
> >> > java version "1.8.0_60"
> >> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> >> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> >> >
> >
> > I am able to properly run the basic Hadoop and Hama examples like the
> > following:
> > bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
> > grep input output 'dfs[a-z.]+'
> > bin/hama jar hama-examples-0.7.0.jar pi
> > My problem is that I receive the following error message when I try to
> run
> > the HelloWorld Hama Streaming example with this instruction:
> > bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> > The default python interpreters of my OS were python-2.7 and python-3.4;
> > since I had problems with this example I also tried to install python-3.2
> > with the following instructions but it didn't solved the problem:
> >
> >
> >>
> >> > sudo apt-get install software-properties-common
> >> > sudo apt-add-repository ppa:fkrull/deadsnakes
> >> > sudo apt-get update
> >> > sudo apt-get install python3.2
> >> >
> >
> >
> > The installed python version is 3.2.6 as you can see from the following
> > lines:
> >
> >
> >>
> >> > $ python3.2
> >> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
> >> > [GCC 4.8.2] on linux2
> >> > Type "help", "copyright", "credits" or "license" for more information.
> >> > >>>
> >> >
> >
> >
> > The error message is the following (I am working in local mode so I
> didn't
> > run bin/start-bspd.sh):
> >
> >
> >>
> >>
> >> >
> >> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> > python3.2 -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> > -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> >> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
> > native-hadoop library for your platform... using builtin-java classes
> where
> > applicable
> >> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name is
> > deprecated. Instead, use fs.defaultFS
> >> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
> > deprecated. Instead, use mapreduce.job.user.name
> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> :an
> > attempt to override final parameter:
> > mapreduce.job.end-notification.max.retry.interval; Ignoring.
> >> > 15/09/30 12:39:11 WARN conf.Configuration:
> > org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80
> :an
> > attempt to override final parameter:
> > mapreduce.job.end-notification.max.attempts; Ignoring.
> >> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
> > deprecated. Instead, use mapreduce.job.user.name
> >> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
> > job_localrunner_0001
> >> > 15/09/30 12:39:12 INFO Configuration.deprecation:
> > mapred.cache.localFiles is deprecated. Instead, use
> > mapreduce.job.cache.local.files
> >> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new barrier
> for
> > 2 tasks!
> >> > java.lang.NumberFormatException: For input string: "Traceback (most
> > recent call last):"
> >> > at
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >> > at java.lang.Integer.parseInt(Integer.java:580)
> >> > at java.lang.Integer.parseInt(Integer.java:615)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >> > java.lang.NumberFormatException: For input string: "Traceback (most
> > recent call last):"
> >> > at
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> >> > at java.lang.Integer.parseInt(Integer.java:580)
> >> > at java.lang.Integer.parseInt(Integer.java:615)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception:
> Bad
> > command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> >
> >> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception:
> Bad
> > command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> >
> >> > java.util.concurrent.BrokenBarrierException
> >> > at
> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >> > at
> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >> > at
> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >> > at
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> > java.lang.Exception: Bad command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >> > Caused by: java.lang.Exception: Bad command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> > java.util.concurrent.BrokenBarrierException
> >> > at
> > java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> >> > at
> > java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> >> > at
> > org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> >> > at
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> > java.lang.Exception: Bad command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> >> > Caused by: java.lang.Exception: Bad command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> > execution!
> >> > java.io.IOException: Stream closed
> >> > at
> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >> > at
> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> > at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> > at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> >> > at
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> > execution!
> >> > java.lang.InterruptedException: java.lang.Exception: Bad command code:
> > -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> >
> >> > at
> >
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> >> > at
> >
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> >> > at
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after
> bsp
> > execution.
> >> > java.io.IOException: Stream closed
> >> > at
> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >> > at
> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> > at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> > at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >> > at
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after
> bsp
> > execution.
> >> > java.io.IOException: Stream closed
> >> > at
> > java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> >> > at java.io.OutputStream.write(OutputStream.java:116)
> >> > at
> > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> >> > at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> >> > at
> > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> >> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> >> > at
> >
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> >> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> >> > at
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> > execution!
> >> > java.util.concurrent.ExecutionException:
> > java.lang.InterruptedException: java.lang.Exception: Bad command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> >
> >> > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> >> > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > Caused by: java.lang.InterruptedException: java.lang.Exception: Bad
> > command code: -2
> >> > at
> > org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> >> >
> >> > at
> >
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> >> > at
> >
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> >> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> >> > at
> > org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >> > at
> >
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >> > ... 1 more
> >> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
> >> >
> >
> >
> > I tried to searching for these error messages with google but I didn't
> find
> > any solution.
> >
> > Does anyone of you have any idea of how can I solve this problem?
> >
> > Any help is appreciated :)
> >
> > Thank you
> > Best
> >
> > Vincenzo
>
>
>
> --
> Best Regards, Edward J. Yoon
>
--
-----------------------------------------------------------------------------
Dr. Vincenzo De Leo, PhD
Research and Development @ Linkalab
-----------------------------------------------------------------------------
Home Page: http://www.linkalab.it/people/vincenzo.deleo
e-mail: vincenzo.deleo@linkalab.it
Phone: (+39)0706753140
Mobile: (+39)3406673544
Skype contact: vincenzo.deleo77
GTalk contact: nagatobimaru
-----------------------------------------------------------------------------
Linkalab (http://www.linkalab.it)
CoMPLeX SySTeMS CoMPuTaTioNaL LaBoRaToRy
Via San Benedetto, 88
09129 Cagliari (CA), ITALY
P. IVA: 03148020922
Phone: (+39) 0707966021
FAX: (+39) 0707962000
-----------------------------------------------------------------------------
Re: hama streaming problem with hello world example
Posted by "Edward J. Yoon" <ed...@apache.org>.
It looks like a bug of input reader. Can you try to run K-Means example?
On Wed, Sep 30, 2015 at 10:17 PM, Vincenzo De Leo
<na...@gmail.com> wrote:
> Hi to all,
> this is my first mail to this mailing list so please have patience if I
> make some bad choice in the format.
>
> I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500 with 2 GB
> of RAM.
>
> I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e Hama-0.7.0
> as you can see from the following lines:
>
>
>>
>> > $ hadoop version
>> > Hadoop 2.7.1
>> > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
> 15ecc87ccf4a0228f35af08fc56de536e6ce657a
>> > Compiled by jenkins on 2015-06-29T06:04Z
>> > Compiled with protoc 2.5.0
>> > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
>> > This command was run using
> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
>
>
>> > $ java -version
>> > java version "1.8.0_60"
>> > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
>> > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>> >
>
> I am able to properly run the basic Hadoop and Hama examples like the
> following:
> bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
> grep input output 'dfs[a-z.]+'
> bin/hama jar hama-examples-0.7.0.jar pi
> My problem is that I receive the following error message when I try to run
> the HelloWorld Hama Streaming example with this instruction:
> bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> The default python interpreters of my OS were python-2.7 and python-3.4;
> since I had problems with this example I also tried to install python-3.2
> with the following instructions but it didn't solved the problem:
>
>
>>
>> > sudo apt-get install software-properties-common
>> > sudo apt-add-repository ppa:fkrull/deadsnakes
>> > sudo apt-get update
>> > sudo apt-get install python3.2
>> >
>
>
> The installed python version is 3.2.6 as you can see from the following
> lines:
>
>
>>
>> > $ python3.2
>> > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
>> > [GCC 4.8.2] on linux2
>> > Type "help", "copyright", "credits" or "license" for more information.
>> > >>>
>> >
>
>
> The error message is the following (I am working in local mode so I didn't
> run bin/start-bspd.sh):
>
>
>>
>>
>> >
>> > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> python3.2 -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>> > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
>> > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
>> > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
>> > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
> deprecated. Instead, use mapreduce.job.user.name
>> > 15/09/30 12:39:11 WARN conf.Configuration:
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
>> > 15/09/30 12:39:11 WARN conf.Configuration:
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
>> > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
> deprecated. Instead, use mapreduce.job.user.name
>> > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
> job_localrunner_0001
>> > 15/09/30 12:39:12 INFO Configuration.deprecation:
> mapred.cache.localFiles is deprecated. Instead, use
> mapreduce.job.cache.local.files
>> > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new barrier for
> 2 tasks!
>> > java.lang.NumberFormatException: For input string: "Traceback (most
> recent call last):"
>> > at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> > at java.lang.Integer.parseInt(Integer.java:580)
>> > at java.lang.Integer.parseInt(Integer.java:615)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> > java.lang.NumberFormatException: For input string: "Traceback (most
> recent call last):"
>> > at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> > at java.lang.Integer.parseInt(Integer.java:580)
>> > at java.lang.Integer.parseInt(Integer.java:615)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
>> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >
>> > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >
>> > java.util.concurrent.BrokenBarrierException
>> > at
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> > at
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> > at
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> java.lang.Exception: Bad command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> > Caused by: java.lang.Exception: Bad command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > java.util.concurrent.BrokenBarrierException
>> > at
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
>> > at
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
>> > at
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
>> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> java.lang.Exception: Bad command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
>> > Caused by: java.lang.Exception: Bad command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
>> > java.io.IOException: Stream closed
>> > at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > at java.io.OutputStream.write(OutputStream.java:116)
>> > at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
>> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
>> > java.lang.InterruptedException: java.lang.Exception: Bad command code:
> -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >
>> > at
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> > at
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> execution.
>> > java.io.IOException: Stream closed
>> > at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > at java.io.OutputStream.write(OutputStream.java:116)
>> > at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> execution.
>> > java.io.IOException: Stream closed
>> > at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
>> > at java.io.OutputStream.write(OutputStream.java:116)
>> > at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>> > at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>> > at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
>> > at java.io.DataOutputStream.flush(DataOutputStream.java:123)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
>> > at
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
>> > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
>> > java.util.concurrent.ExecutionException:
> java.lang.InterruptedException: java.lang.Exception: Bad command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >
>> > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
>> > at java.lang.Thread.run(Thread.java:745)
>> > Caused by: java.lang.InterruptedException: java.lang.Exception: Bad
> command code: -2
>> > at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
>> >
>> > at
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
>> > at
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
>> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>> > at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > ... 1 more
>> > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
>> >
>
>
> I tried to searching for these error messages with google but I didn't find
> any solution.
>
> Does anyone of you have any idea of how can I solve this problem?
>
> Any help is appreciated :)
>
> Thank you
> Best
>
> Vincenzo
--
Best Regards, Edward J. Yoon