You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by Sandy Ding <sa...@gmail.com> on 2012/09/11 13:54:04 UTC

my new hama cluster cannot run pi example

Hi, all,

I newly set up a 3-node hama cluster following the HamaInstallationGuide.
But I got some confusing errors when running the pi examples, the tasklog
is as follows:

12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=brick4:
21810,r910:21810 sessionTimeout=1200000
watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
l@e61a35
12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket connection to
server brick4/10.131.
201.14:21810
12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
Zookeeper! At r910.ppi
/10.131.201.90:61001
12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection established
to brick4/10.131.201
.14:21810, initiating session
12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session establishment complete
on server brick4/10
.131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated timeout =
1200000
12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to brick4.r715/
10.131.201.14:61003
12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] OPEN
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
tSocketPipelineSink.java:384)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
eys(NioClientSocketPipelineSink.java:354)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
ketPipelineSink.java:276)
        at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] CLOSED
12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer brick4.r715/
10.131.201.14:61003 closed
 connection.
12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and bsp
function.
java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
        at
org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
        at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
        at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
ava:83)
        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
        at
org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
        at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
        at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
        at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception from
downstream.
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
 at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session: 0x139b47286bf000c
closed
12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut down
12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
        at
org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
        at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
        at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
        at
org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
        at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
        at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
        at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
        at
org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
        at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
        at
org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
        at
org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
        at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
        at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
        at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
        at
org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
.....

Can anybody help? I am really desparate...

Thanks in advance,

Sandy

Re: Re: my new hama cluster cannot run pi example

Posted by Thomas Jungblut <th...@gmail.com>.
If your file is in HDFS, and you have configured the HDFS in your configs
then it will search in HDFS first. No matter what you tell him of a scheme.
The hdfs scheme is hdfs:// instead of file://.



2012/9/12 Sandy Ding <sa...@gmail.com>

> Yes, because I don't know how to specify the input file if I use hdfs.
> Can I manually (instead of calling the open() and write API of hdfs) write
> files in hdfs?
>
> 2012/9/12 钱昊 <qh...@126.com>
>
> > Do you want to use local file system insteadof hdfs?
> >
> >
> > At 2012-09-12 20:02:35,"Sandy Ding" <sa...@gmail.com> wrote:
> > >Sorry about the incomplete information.
> > >I have changed the default filesystem to "file:///" instead of "local"
> > >But the error still occurred:
> > >   Error reading task
> >
> >
> >outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
> > >Anything special to do if the default filesystem is set to lcoal?
> > >
> > >2012/9/12 Sandy Ding <sa...@gmail.com>
> > >
> > >> Yet another problem occurred...
> > >> The Installation Guide says the value for fs.default.name is either
> > local
> > >> or host:port for HDFS.
> > >> But when I set it to "local" to run pagerank, the system kept
> complain :
> > >> "local is  "
> > >>  Error reading task
> > >>
> >
> outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
> > >>
> > >>
> > >> 2012/9/12 Thomas Jungblut <th...@gmail.com>
> > >>
> > >>> Hey,
> > >>>
> > >>> it now takes plain text as input. Sorry the wikipedia page is
> > deprecated,
> > >>> will update it then.
> > >>>
> > >>> The format should be:
> > >>>
> > >>> VERTEX_ID\t(n-tab separated VERTEX_IDs) \n
> > >>> E.g:
> > >>> 1\t2\t3\t4 // makes 1, adjacent to 2, 3 and 4
> > >>> 2\t3\t1
> > >>>
> > >>> 2012/9/12 Sandy Ding <sa...@gmail.com>
> > >>>
> > >>> > Hi,
> > >>> >
> > >>> > The pagerank-text2seq seems to have been removed from the
> > >>> 0.5.0-examples?
> > >>> > I got the error:
> > >>> >
> > >>> > Unknown program 'pagerank-text2seq' chosen.
> > >>> >
> > >>> > So how should I run the pagerank example now?
> > >>> >
> > >>> >
> > >>> > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >>> >
> > >>> > > On the machine where you submit the job, you can delete the
> > >>> > > "hama.messenger.compression.class" property in your default
> > >>> > configuration.
> > >>> > > This should null out the compressor and make it run without
> > >>> compression.
> > >>> > >
> > >>> > >
> > >>> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >>> > >
> > >>> > > > Plus, how to remove the snappy compression form the
> > configuration?
> > >>> > > >
> > >>> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >>> > > >
> > >>> > > > > It actually failed on another node brick5 too.
> > >>> > > > > I do have the write permission of /tmp dir.
> > >>> > > > > But the most weird thing is:
> > >>> > > > >  the snappy..so file is generated every time I run the
> example,
> > >>> but
> > >>> > its
> > >>> > > > > owner is another user who has once run hama on the same
> > machine,
> > >>> even
> > >>> > > > when
> > >>> > > > > I manually rm and re-generate the file! And I don't have the
> > >>> > permission
> > >>> > > > to
> > >>> > > > > rm it!
> > >>> > > > >
> > >>> > > > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >>> > > > >
> > >>> > > > >> Oh, or just remove the snappy compression from the
> > configuration
> > >>> ;)
> > >>> > > > >>
> > >>> > > > >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >>> > > > >>
> > >>> > > > >> > Can you give him permission to delete in /tmp/?
> > >>> > > > >> > Why is it not failing on the other hosts?
> > >>> > > > >> > Otherwise you have to manually add the snappy library to
> the
> > >>> lib
> > >>> > > path
> > >>> > > > of
> > >>> > > > >> > the task.
> > >>> > > > >> >
> > >>> > > > >> >
> > >>> > > > >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >>> > > > >> >
> > >>> > > > >> >> the IOException stack:
> > >>> > > > >> >>
> > >>> > > > >> >> java.io.IOException: failed to remove existing native
> > library
> > >>> > file:
> > >>> > > > >> >> /tmp/snappy-1.0.4.1-libsna
> > >>> > > > >> >> ppyjava.so
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > >
> > >>>
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > >
> > >>>
> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > >
> > >>>
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
> > >>> > > > >> >>         at
> > >>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > >>> > > > >> >>         at
> > org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > >
> > >>>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > >
> > >>>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> > >>> > > > >> >> or.java:43)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> > >>> > > > >> >> gerImpl.java:135)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> > >>> > > > >> >> java:79)
> > >>> > > > >> >>         at
> > >>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> >
> > >>>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >>> > > > >> >>         at
> > >>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >>> > > > >> >>         at
> > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > >
> > >>>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >>> > > > >> >> java.lang.reflect.InvocationTargetException
> > >>> > > > >> >>         at
> > sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >>> > > > Method)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> > >>> > > > >> >> 5)
> > >>> > > > >> >>         at
> java.lang.reflect.Method.invoke(Method.java:597)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > >
> > >>>
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> > >>> > > > >> >>         at
> > >>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > >>> > > > >> >>         at
> > org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > >
> > >>>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > >
> > >>>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > >>> > > > >> >>         at
> > >>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> >
> > >>>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >>> > > > >> >>         at
> > >>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >>> > > > >> >>         at
> > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > >
> > >>>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >>> > > > >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava
> in
> > >>> > > > >> >> java.library.path
> > >>> > > > >> >>         at
> > >>> > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> > >>> > > > >> >>         at
> java.lang.Runtime.loadLibrary0(Runtime.java:823)
> > >>> > > > >> >>         at java.lang.System.loadLibrary(System.java:1028)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> > >>> > > > >> >>         ... 17 more
> > >>> > > > >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp
> > setup
> > >>> and
> > >>> > > bsp
> > >>> > > > >> >> function.
> > >>> > > > >> >> java.lang.NullPointerException
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > >>> > > > >> >>         at
> > >>> > > > >> >>
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > >>> > > > >> >>         at
> > >>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >>> > > > >> >>
> > >>> > > > >> >> the user that I currently run hama doesn't have
> >  permissions
> > >>> to
> > >>> > > > remove
> > >>> > > > >> >> /tmp/.  files.
> > >>> > > > >> >> but I cannot run hama using sudo.
> > >>> > > > >> >> doesn't what to do...
> > >>> > > > >> >>
> > >>> > > > >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >>> > > > >> >>
> > >>> > > > >> >> > If you don't have additional stack information it is
> > >>> difficult
> > >>> > to
> > >>> > > > >> guess.
> > >>> > > > >> >> > According to
> > >>> > > > >> >> >
> > >>> > > >
> > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> > >>> > > > >> >> > it should throw an IOException that contains the
> > underlying
> > >>> > > error.
> > >>> > > > If
> > >>> > > > >> >> the
> > >>> > > > >> >> > shared object can't be deleted, I would try to start by
> > >>> > checking
> > >>> > > > >> rights
> > >>> > > > >> >> or
> > >>> > > > >> >> > what on the other machines is not like on the machine
> > that
> > >>> > > failed.
> > >>> > > > >> >> >
> > >>> > > > >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >>> > > > >> >> >
> > >>> > > > >> >> > > Any possible reasons for that?
> > >>> > > > >> >> > >
> > >>> > > > >> >> > > 2012/9/11 Thomas Jungblut <thomas.jungblut@gmail.com
> >
> > >>> > > > >> >> > >
> > >>> > > > >> >> > > > Hi,
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > > > that is actually a good question.
> > >>> > > > >> >> > > > According to the exception and the code [1] it was
> > >>> unable
> > >>> > to
> > >>> > > > >> delete
> > >>> > > > >> >> the
> > >>> > > > >> >> > > > file.
> > >>> > > > >> >> > > > Actually this could have various reasons, what
> > operating
> > >>> > > system
> > >>> > > > >> are
> > >>> > > > >> >> you
> > >>> > > > >> >> > > > running?
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > > > [1]
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > > > > What actually happened in brick4 (the node that
> > >>> refused
> > >>> > > > >> >> > r910(master)'s
> > >>> > > > >> >> > > > > connection) is that it
> > >>> > > > >> >> > > > > " failed to remove existing native library file:
> > /tmp/
> > >>> > > > >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
> > >>> > > > >> >> > > > >         at
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> >
> > >>>
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > >>> > > > >> >> > > > > then
> > >>> > > > >> >> > > > > java.lang.NullPointerException
> > >>> > > > >> >> > > > >         at
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > > > But I actually run in the Hama directory and
> > >>> > > > >> >> snappy-java-1.0.4.1.jar
> > >>> > > > >> >> > is
> > >>> > > > >> >> > > > > included in hama/lib dir.
> > >>> > > > >> >> > > > > I even include the
> > hama/lib/snappy-java-1.0.4.1.jar in
> > >>> > > > >> CLASSPATH.
> > >>> > > > >> >> > > > > What happened?
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > > > > Hi, all,
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > > > I newly set up a 3-node hama cluster following
> > the
> > >>> > > > >> >> > > > HamaInstallationGuide.
> > >>> > > > >> >> > > > > > But I got some confusing errors when running
> the
> > pi
> > >>> > > > examples,
> > >>> > > > >> >> the
> > >>> > > > >> >> > > > tasklog
> > >>> > > > >> >> > > > > > is as follows:
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper:
> > >>> Initiating
> > >>> > > > client
> > >>> > > > >> >> > > > connection,
> > >>> > > > >> >> > > > > > connectString=brick4:
> > >>> > > > >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
> > >>> > > > >> >> > > > > >
> > >>> watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > >>> > > > >> >> > > > > > l@e61a35
> > >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn:
> > Opening
> > >>> > > socket
> > >>> > > > >> >> > > connection
> > >>> > > > >> >> > > > to
> > >>> > > > >> >> > > > > > server brick4/10.131.
> > >>> > > > >> >> > > > > > 201.14:21810
> > >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO
> > sync.ZooKeeperSyncClientImpl:
> > >>> > > Start
> > >>> > > > >> >> > connecting
> > >>> > > > >> >> > > > to
> > >>> > > > >> >> > > > > > Zookeeper! At r910.ppi
> > >>> > > > >> >> > > > > > /10.131.201.90:61001
> > >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn:
> > Socket
> > >>> > > > >> connection
> > >>> > > > >> >> > > > > established
> > >>> > > > >> >> > > > > > to brick4/10.131.201
> > >>> > > > >> >> > > > > > .14:21810, initiating session
> > >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn:
> > Session
> > >>> > > > >> >> establishment
> > >>> > > > >> >> > > > > > complete on server brick4/10
> > >>> > > > >> >> > > > > > .131.201.14:21810, sessionid =
> 0x139b47286bf000c,
> > >>> > > > negotiated
> > >>> > > > >> >> > timeout
> > >>> > > > >> >> > > =
> > >>> > > > >> >> > > > > > 1200000
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
> > >>> Connecting
> > >>> > > to
> > >>> > > > >> >> > > brick4.r715/
> > >>> > > > >> >> > > > > > 10.131.201.14:61003
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
> [id:
> > >>> > > > 0x0118278a]
> > >>> > > > >> >> OPEN
> > >>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
> > >>> > > > >> >> > > > > >         at
> > >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >>> > > > >> >> Method)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > >
> > >>> > > > >>
> > >>> >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > >>> > > > >> >> > > > > > tSocketPipelineSink.java:384)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > >>> > > > >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > >>> > > > >> >> > > > > > ketPipelineSink.java:276)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >>> > > > >> >> > > > > >         at
> java.lang.Thread.run(Thread.java:662)
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
> [id:
> > >>> > > > 0x0118278a]
> > >>> > > > >> >> > CLOSED
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
> > Remote
> > >>> > peer
> > >>> > > > >> >> > brick4.r715/
> > >>> > > > >> >> > > > > > 10.131.201.14:61003 closed
> > >>> > > > >> >> > > > > >  connection.
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error
> > running
> > >>> bsp
> > >>> > > > setup
> > >>> > > > >> and
> > >>> > > > >> >> > bsp
> > >>> > > > >> >> > > > > > function.
> > >>> > > > >> >> > > > > > java.io.IOException: Error connecting to
> > >>> brick4.r715/
> > >>> > > > >> >> > > > 10.131.201.14:61003
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > >
> > >>> > > > >>
> > >>> >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> >
> > >>> > > >
> > >>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > >>> > > > >> >> > > > > > ava:83)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> >
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> >
> > >>>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> >
> > >>> > > > >>
> > >>> > >
> > >>>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >>> > > > >> >> > > > > > Caused by: java.net.ConnectException:
> Connection
> > >>> > refused
> > >>> > > > >> >> > > > > >         at
> > >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >>> > > > >> >> Method)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > >
> > >>> > > > >>
> > >>> >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >>> > > > >> >> > > > > >         at
> java.lang.Thread.run(Thread.java:662)
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver:
> > >>> Unexpected
> > >>> > > > >> >> exception
> > >>> > > > >> >> > > from
> > >>> > > > >> >> > > > > > downstream.
> > >>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
> > >>> > > > >> >> > > > > >         at
> > >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >>> > > > >> >> Method)
> > >>> > > > >> >> > > > > >  at
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> >
> > >>> > > > >>
> > >>> >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >>> > > > >> >> > > > > >         at
> java.lang.Thread.run(Thread.java:662)
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper:
> > Session:
> > >>> > > > >> >> > > 0x139b47286bf000c
> > >>> > > > >> >> > > > > > closed
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn:
> > >>> > EventThread
> > >>> > > > shut
> > >>> > > > >> >> down
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting
> > down
> > >>> ping
> > >>> > > > >> service.
> > >>> > > > >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error
> > >>> running
> > >>> > > > child
> > >>> > > > >> >> > > > > > java.io.IOException: Error connecting to
> > >>> brick4.r715/
> > >>> > > > >> >> > > > 10.131.201.14:61003
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > >
> > >>> > > > >>
> > >>> >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> >
> > >>> > > >
> > >>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> >
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> >
> > >>>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> >
> > >>> > > > >>
> > >>> > >
> > >>>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >>> > > > >> >> > > > > > Caused by: java.net.ConnectException:
> Connection
> > >>> > refused
> > >>> > > > >> >> > > > > >         at
> > >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >>> > > > >> >> Method)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > >
> > >>> > > > >>
> > >>> >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >>> > > > >> >> > > > > >         at
> java.lang.Thread.run(Thread.java:662)
> > >>> > > > >> >> > > > > > java.io.IOException: Error connecting to
> > >>> brick4.r715/
> > >>> > > > >> >> > > > 10.131.201.14:61003
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > >
> > >>> > > > >>
> > >>> >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> >
> > >>> > > >
> > >>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> >
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> >
> > >>>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> >
> > >>> > > > >>
> > >>> > >
> > >>>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >>> > > > >> >> > > > > > Caused by: java.net.ConnectException:
> Connection
> > >>> > refused
> > >>> > > > >> >> > > > > >         at
> > >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >>> > > > >> >> Method)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > >
> > >>> > > > >>
> > >>> >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >>> > > > >> >> > > > > >         at
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >>> > > > >> >> > > > > > .....
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > > > Can anybody help? I am really desparate...
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > > > Thanks in advance,
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > > > Sandy
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > > >
> > >>> > > > >> >> > > > >
> > >>> > > > >> >> > > >
> > >>> > > > >> >> > >
> > >>> > > > >> >> >
> > >>> > > > >> >>
> > >>> > > > >> >
> > >>> > > > >> >
> > >>> > > > >>
> > >>> > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> >
>

Re: Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
Yes, because I don't know how to specify the input file if I use hdfs.
Can I manually (instead of calling the open() and write API of hdfs) write
files in hdfs?

2012/9/12 钱昊 <qh...@126.com>

> Do you want to use local file system insteadof hdfs?
>
>
> At 2012-09-12 20:02:35,"Sandy Ding" <sa...@gmail.com> wrote:
> >Sorry about the incomplete information.
> >I have changed the default filesystem to "file:///" instead of "local"
> >But the error still occurred:
> >   Error reading task
>
> >outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
> >Anything special to do if the default filesystem is set to lcoal?
> >
> >2012/9/12 Sandy Ding <sa...@gmail.com>
> >
> >> Yet another problem occurred...
> >> The Installation Guide says the value for fs.default.name is either
> local
> >> or host:port for HDFS.
> >> But when I set it to "local" to run pagerank, the system kept complain :
> >> "local is  "
> >>  Error reading task
> >>
> outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
> >>
> >>
> >> 2012/9/12 Thomas Jungblut <th...@gmail.com>
> >>
> >>> Hey,
> >>>
> >>> it now takes plain text as input. Sorry the wikipedia page is
> deprecated,
> >>> will update it then.
> >>>
> >>> The format should be:
> >>>
> >>> VERTEX_ID\t(n-tab separated VERTEX_IDs) \n
> >>> E.g:
> >>> 1\t2\t3\t4 // makes 1, adjacent to 2, 3 and 4
> >>> 2\t3\t1
> >>>
> >>> 2012/9/12 Sandy Ding <sa...@gmail.com>
> >>>
> >>> > Hi,
> >>> >
> >>> > The pagerank-text2seq seems to have been removed from the
> >>> 0.5.0-examples?
> >>> > I got the error:
> >>> >
> >>> > Unknown program 'pagerank-text2seq' chosen.
> >>> >
> >>> > So how should I run the pagerank example now?
> >>> >
> >>> >
> >>> > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >>> >
> >>> > > On the machine where you submit the job, you can delete the
> >>> > > "hama.messenger.compression.class" property in your default
> >>> > configuration.
> >>> > > This should null out the compressor and make it run without
> >>> compression.
> >>> > >
> >>> > >
> >>> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >>> > >
> >>> > > > Plus, how to remove the snappy compression form the
> configuration?
> >>> > > >
> >>> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >>> > > >
> >>> > > > > It actually failed on another node brick5 too.
> >>> > > > > I do have the write permission of /tmp dir.
> >>> > > > > But the most weird thing is:
> >>> > > > >  the snappy..so file is generated every time I run the example,
> >>> but
> >>> > its
> >>> > > > > owner is another user who has once run hama on the same
> machine,
> >>> even
> >>> > > > when
> >>> > > > > I manually rm and re-generate the file! And I don't have the
> >>> > permission
> >>> > > > to
> >>> > > > > rm it!
> >>> > > > >
> >>> > > > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >>> > > > >
> >>> > > > >> Oh, or just remove the snappy compression from the
> configuration
> >>> ;)
> >>> > > > >>
> >>> > > > >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >>> > > > >>
> >>> > > > >> > Can you give him permission to delete in /tmp/?
> >>> > > > >> > Why is it not failing on the other hosts?
> >>> > > > >> > Otherwise you have to manually add the snappy library to the
> >>> lib
> >>> > > path
> >>> > > > of
> >>> > > > >> > the task.
> >>> > > > >> >
> >>> > > > >> >
> >>> > > > >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >>> > > > >> >
> >>> > > > >> >> the IOException stack:
> >>> > > > >> >>
> >>> > > > >> >> java.io.IOException: failed to remove existing native
> library
> >>> > file:
> >>> > > > >> >> /tmp/snappy-1.0.4.1-libsna
> >>> > > > >> >> ppyjava.so
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >>
> >>> > >
> >>>
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > >
> >>> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > >
> >>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
> >>> > > > >> >>         at
> >>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> >>> > > > >> >>         at
> org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > >
> >>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > >
> >>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> >>> > > > >> >> or.java:43)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> >>> > > > >> >> gerImpl.java:135)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> >>> > > > >> >> java:79)
> >>> > > > >> >>         at
> >>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> >
> >>>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >>> > > > >> >>         at
> >>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >>> > > > >> >>         at
> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >>
> >>> > >
> >>>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >>> > > > >> >> java.lang.reflect.InvocationTargetException
> >>> > > > >> >>         at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >>> > > > Method)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> >>> > > > >> >> 5)
> >>> > > > >> >>         at java.lang.reflect.Method.invoke(Method.java:597)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > >
> >>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> >>> > > > >> >>         at
> >>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> >>> > > > >> >>         at
> org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > >
> >>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > >
> >>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> >>> > > > >> >>         at
> >>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> >
> >>>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >>> > > > >> >>         at
> >>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >>> > > > >> >>         at
> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >>
> >>> > >
> >>>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >>> > > > >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
> >>> > > > >> >> java.library.path
> >>> > > > >> >>         at
> >>> > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> >>> > > > >> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
> >>> > > > >> >>         at java.lang.System.loadLibrary(System.java:1028)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> >>> > > > >> >>         ... 17 more
> >>> > > > >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp
> setup
> >>> and
> >>> > > bsp
> >>> > > > >> >> function.
> >>> > > > >> >> java.lang.NullPointerException
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> >>> > > > >> >>         at
> >>> > > > >> >>
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> >>> > > > >> >>         at
> >>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>> > > > >> >>
> >>> > > > >> >> the user that I currently run hama doesn't have
>  permissions
> >>> to
> >>> > > > remove
> >>> > > > >> >> /tmp/.  files.
> >>> > > > >> >> but I cannot run hama using sudo.
> >>> > > > >> >> doesn't what to do...
> >>> > > > >> >>
> >>> > > > >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >>> > > > >> >>
> >>> > > > >> >> > If you don't have additional stack information it is
> >>> difficult
> >>> > to
> >>> > > > >> guess.
> >>> > > > >> >> > According to
> >>> > > > >> >> >
> >>> > > >
> http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> >>> > > > >> >> > it should throw an IOException that contains the
> underlying
> >>> > > error.
> >>> > > > If
> >>> > > > >> >> the
> >>> > > > >> >> > shared object can't be deleted, I would try to start by
> >>> > checking
> >>> > > > >> rights
> >>> > > > >> >> or
> >>> > > > >> >> > what on the other machines is not like on the machine
> that
> >>> > > failed.
> >>> > > > >> >> >
> >>> > > > >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >>> > > > >> >> >
> >>> > > > >> >> > > Any possible reasons for that?
> >>> > > > >> >> > >
> >>> > > > >> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >>> > > > >> >> > >
> >>> > > > >> >> > > > Hi,
> >>> > > > >> >> > > >
> >>> > > > >> >> > > > that is actually a good question.
> >>> > > > >> >> > > > According to the exception and the code [1] it was
> >>> unable
> >>> > to
> >>> > > > >> delete
> >>> > > > >> >> the
> >>> > > > >> >> > > > file.
> >>> > > > >> >> > > > Actually this could have various reasons, what
> operating
> >>> > > system
> >>> > > > >> are
> >>> > > > >> >> you
> >>> > > > >> >> > > > running?
> >>> > > > >> >> > > >
> >>> > > > >> >> > > > [1]
> >>> > > > >> >> > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> >>> > > > >> >> > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >>> > > > >> >> > > >
> >>> > > > >> >> > > > > What actually happened in brick4 (the node that
> >>> refused
> >>> > > > >> >> > r910(master)'s
> >>> > > > >> >> > > > > connection) is that it
> >>> > > > >> >> > > > > " failed to remove existing native library file:
> /tmp/
> >>> > > > >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
> >>> > > > >> >> > > > >         at
> >>> > > > >> >> > > > >
> >>> > > > >> >> > >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> >
> >>>
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> >>> > > > >> >> > > > > then
> >>> > > > >> >> > > > > java.lang.NullPointerException
> >>> > > > >> >> > > > >         at
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > > > But I actually run in the Hama directory and
> >>> > > > >> >> snappy-java-1.0.4.1.jar
> >>> > > > >> >> > is
> >>> > > > >> >> > > > > included in hama/lib dir.
> >>> > > > >> >> > > > > I even include the
> hama/lib/snappy-java-1.0.4.1.jar in
> >>> > > > >> CLASSPATH.
> >>> > > > >> >> > > > > What happened?
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > > > > Hi, all,
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > > > I newly set up a 3-node hama cluster following
> the
> >>> > > > >> >> > > > HamaInstallationGuide.
> >>> > > > >> >> > > > > > But I got some confusing errors when running the
> pi
> >>> > > > examples,
> >>> > > > >> >> the
> >>> > > > >> >> > > > tasklog
> >>> > > > >> >> > > > > > is as follows:
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper:
> >>> Initiating
> >>> > > > client
> >>> > > > >> >> > > > connection,
> >>> > > > >> >> > > > > > connectString=brick4:
> >>> > > > >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
> >>> > > > >> >> > > > > >
> >>> watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> >>> > > > >> >> > > > > > l@e61a35
> >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn:
> Opening
> >>> > > socket
> >>> > > > >> >> > > connection
> >>> > > > >> >> > > > to
> >>> > > > >> >> > > > > > server brick4/10.131.
> >>> > > > >> >> > > > > > 201.14:21810
> >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO
> sync.ZooKeeperSyncClientImpl:
> >>> > > Start
> >>> > > > >> >> > connecting
> >>> > > > >> >> > > > to
> >>> > > > >> >> > > > > > Zookeeper! At r910.ppi
> >>> > > > >> >> > > > > > /10.131.201.90:61001
> >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn:
> Socket
> >>> > > > >> connection
> >>> > > > >> >> > > > > established
> >>> > > > >> >> > > > > > to brick4/10.131.201
> >>> > > > >> >> > > > > > .14:21810, initiating session
> >>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn:
> Session
> >>> > > > >> >> establishment
> >>> > > > >> >> > > > > > complete on server brick4/10
> >>> > > > >> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c,
> >>> > > > negotiated
> >>> > > > >> >> > timeout
> >>> > > > >> >> > > =
> >>> > > > >> >> > > > > > 1200000
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
> >>> Connecting
> >>> > > to
> >>> > > > >> >> > > brick4.r715/
> >>> > > > >> >> > > > > > 10.131.201.14:61003
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> >>> > > > 0x0118278a]
> >>> > > > >> >> OPEN
> >>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
> >>> > > > >> >> > > > > >         at
> >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >>> > > > >> >> Method)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > >
> >>> > > > >>
> >>> >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> >>> > > > >> >> > > > > > tSocketPipelineSink.java:384)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> >>> > > > >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> >>> > > > >> >> > > > > > ketPipelineSink.java:276)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> >>> > > > 0x0118278a]
> >>> > > > >> >> > CLOSED
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
> Remote
> >>> > peer
> >>> > > > >> >> > brick4.r715/
> >>> > > > >> >> > > > > > 10.131.201.14:61003 closed
> >>> > > > >> >> > > > > >  connection.
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error
> running
> >>> bsp
> >>> > > > setup
> >>> > > > >> and
> >>> > > > >> >> > bsp
> >>> > > > >> >> > > > > > function.
> >>> > > > >> >> > > > > > java.io.IOException: Error connecting to
> >>> brick4.r715/
> >>> > > > >> >> > > > 10.131.201.14:61003
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > >
> >>> > > > >>
> >>> >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> >
> >>> > > >
> >>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> >>> > > > >> >> > > > > > ava:83)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> >
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> >
> >>>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >>> > > > >> >> > > > > >         at
> >>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> >
> >>> > > > >>
> >>> > >
> >>>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> >>> > refused
> >>> > > > >> >> > > > > >         at
> >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >>> > > > >> >> Method)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > >
> >>> > > > >>
> >>> >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver:
> >>> Unexpected
> >>> > > > >> >> exception
> >>> > > > >> >> > > from
> >>> > > > >> >> > > > > > downstream.
> >>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
> >>> > > > >> >> > > > > >         at
> >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >>> > > > >> >> Method)
> >>> > > > >> >> > > > > >  at
> >>> > > > >> >> > > > >
> >>> > > > >> >> >
> >>> > > > >>
> >>> >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper:
> Session:
> >>> > > > >> >> > > 0x139b47286bf000c
> >>> > > > >> >> > > > > > closed
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn:
> >>> > EventThread
> >>> > > > shut
> >>> > > > >> >> down
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting
> down
> >>> ping
> >>> > > > >> service.
> >>> > > > >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error
> >>> running
> >>> > > > child
> >>> > > > >> >> > > > > > java.io.IOException: Error connecting to
> >>> brick4.r715/
> >>> > > > >> >> > > > 10.131.201.14:61003
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > >
> >>> > > > >>
> >>> >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> >
> >>> > > >
> >>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> >
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> >
> >>>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >>> > > > >> >> > > > > >         at
> >>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> >
> >>> > > > >>
> >>> > >
> >>>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> >>> > refused
> >>> > > > >> >> > > > > >         at
> >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >>> > > > >> >> Method)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > >
> >>> > > > >>
> >>> >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >>> > > > >> >> > > > > > java.io.IOException: Error connecting to
> >>> brick4.r715/
> >>> > > > >> >> > > > 10.131.201.14:61003
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > >
> >>> > > > >>
> >>> >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> >
> >>> > > >
> >>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> >
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> >
> >>>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >>> > > > >> >> > > > > >         at
> >>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> >
> >>> > > > >>
> >>> > >
> >>>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> >>> > refused
> >>> > > > >> >> > > > > >         at
> >>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >>> > > > >> >> Method)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > >
> >>> > > > >>
> >>> >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >>> > > > >> >> > > > > >         at
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>> > > > >> >> > > > > > .....
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > > > Can anybody help? I am really desparate...
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > > > Thanks in advance,
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > > > Sandy
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > > >
> >>> > > > >> >> > > > >
> >>> > > > >> >> > > >
> >>> > > > >> >> > >
> >>> > > > >> >> >
> >>> > > > >> >>
> >>> > > > >> >
> >>> > > > >> >
> >>> > > > >>
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
>

Re:Re: my new hama cluster cannot run pi example

Posted by 钱昊 <qh...@126.com>.
Do you want to use local file system insteadof hdfs?


At 2012-09-12 20:02:35,"Sandy Ding" <sa...@gmail.com> wrote:
>Sorry about the incomplete information.
>I have changed the default filesystem to "file:///" instead of "local"
>But the error still occurred:
>   Error reading task
>outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
>Anything special to do if the default filesystem is set to lcoal?
>
>2012/9/12 Sandy Ding <sa...@gmail.com>
>
>> Yet another problem occurred...
>> The Installation Guide says the value for fs.default.name is either local
>> or host:port for HDFS.
>> But when I set it to "local" to run pagerank, the system kept complain :
>> "local is  "
>>  Error reading task
>> outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
>>
>>
>> 2012/9/12 Thomas Jungblut <th...@gmail.com>
>>
>>> Hey,
>>>
>>> it now takes plain text as input. Sorry the wikipedia page is deprecated,
>>> will update it then.
>>>
>>> The format should be:
>>>
>>> VERTEX_ID\t(n-tab separated VERTEX_IDs) \n
>>> E.g:
>>> 1\t2\t3\t4 // makes 1, adjacent to 2, 3 and 4
>>> 2\t3\t1
>>>
>>> 2012/9/12 Sandy Ding <sa...@gmail.com>
>>>
>>> > Hi,
>>> >
>>> > The pagerank-text2seq seems to have been removed from the
>>> 0.5.0-examples?
>>> > I got the error:
>>> >
>>> > Unknown program 'pagerank-text2seq' chosen.
>>> >
>>> > So how should I run the pagerank example now?
>>> >
>>> >
>>> > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>>> >
>>> > > On the machine where you submit the job, you can delete the
>>> > > "hama.messenger.compression.class" property in your default
>>> > configuration.
>>> > > This should null out the compressor and make it run without
>>> compression.
>>> > >
>>> > >
>>> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>>> > >
>>> > > > Plus, how to remove the snappy compression form the configuration?
>>> > > >
>>> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>>> > > >
>>> > > > > It actually failed on another node brick5 too.
>>> > > > > I do have the write permission of /tmp dir.
>>> > > > > But the most weird thing is:
>>> > > > >  the snappy..so file is generated every time I run the example,
>>> but
>>> > its
>>> > > > > owner is another user who has once run hama on the same machine,
>>> even
>>> > > > when
>>> > > > > I manually rm and re-generate the file! And I don't have the
>>> > permission
>>> > > > to
>>> > > > > rm it!
>>> > > > >
>>> > > > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>>> > > > >
>>> > > > >> Oh, or just remove the snappy compression from the configuration
>>> ;)
>>> > > > >>
>>> > > > >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>>> > > > >>
>>> > > > >> > Can you give him permission to delete in /tmp/?
>>> > > > >> > Why is it not failing on the other hosts?
>>> > > > >> > Otherwise you have to manually add the snappy library to the
>>> lib
>>> > > path
>>> > > > of
>>> > > > >> > the task.
>>> > > > >> >
>>> > > > >> >
>>> > > > >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
>>> > > > >> >
>>> > > > >> >> the IOException stack:
>>> > > > >> >>
>>> > > > >> >> java.io.IOException: failed to remove existing native library
>>> > file:
>>> > > > >> >> /tmp/snappy-1.0.4.1-libsna
>>> > > > >> >> ppyjava.so
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >>
>>> > >
>>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > >
>>> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > >
>>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
>>> > > > >> >>         at
>>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>>> > > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > >
>>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > >
>>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
>>> > > > >> >> or.java:43)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
>>> > > > >> >> gerImpl.java:135)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
>>> > > > >> >> java:79)
>>> > > > >> >>         at
>>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> >
>>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>>> > > > >> >>         at
>>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>> > > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >>
>>> > >
>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>>> > > > >> >> java.lang.reflect.InvocationTargetException
>>> > > > >> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> > > > Method)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
>>> > > > >> >> 5)
>>> > > > >> >>         at java.lang.reflect.Method.invoke(Method.java:597)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > >
>>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
>>> > > > >> >>         at
>>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>>> > > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > >
>>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > >
>>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>>> > > > >> >>         at
>>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> >
>>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>>> > > > >> >>         at
>>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>> > > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >>
>>> > >
>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>>> > > > >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
>>> > > > >> >> java.library.path
>>> > > > >> >>         at
>>> > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
>>> > > > >> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
>>> > > > >> >>         at java.lang.System.loadLibrary(System.java:1028)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
>>> > > > >> >>         ... 17 more
>>> > > > >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup
>>> and
>>> > > bsp
>>> > > > >> >> function.
>>> > > > >> >> java.lang.NullPointerException
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>>> > > > >> >>         at
>>> > > > >> >>
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>>> > > > >> >>         at
>>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>> > > > >> >>
>>> > > > >> >> the user that I currently run hama doesn't have  permissions
>>> to
>>> > > > remove
>>> > > > >> >> /tmp/.  files.
>>> > > > >> >> but I cannot run hama using sudo.
>>> > > > >> >> doesn't what to do...
>>> > > > >> >>
>>> > > > >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>>> > > > >> >>
>>> > > > >> >> > If you don't have additional stack information it is
>>> difficult
>>> > to
>>> > > > >> guess.
>>> > > > >> >> > According to
>>> > > > >> >> >
>>> > > > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
>>> > > > >> >> > it should throw an IOException that contains the underlying
>>> > > error.
>>> > > > If
>>> > > > >> >> the
>>> > > > >> >> > shared object can't be deleted, I would try to start by
>>> > checking
>>> > > > >> rights
>>> > > > >> >> or
>>> > > > >> >> > what on the other machines is not like on the machine that
>>> > > failed.
>>> > > > >> >> >
>>> > > > >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
>>> > > > >> >> >
>>> > > > >> >> > > Any possible reasons for that?
>>> > > > >> >> > >
>>> > > > >> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>>> > > > >> >> > >
>>> > > > >> >> > > > Hi,
>>> > > > >> >> > > >
>>> > > > >> >> > > > that is actually a good question.
>>> > > > >> >> > > > According to the exception and the code [1] it was
>>> unable
>>> > to
>>> > > > >> delete
>>> > > > >> >> the
>>> > > > >> >> > > > file.
>>> > > > >> >> > > > Actually this could have various reasons, what operating
>>> > > system
>>> > > > >> are
>>> > > > >> >> you
>>> > > > >> >> > > > running?
>>> > > > >> >> > > >
>>> > > > >> >> > > > [1]
>>> > > > >> >> > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
>>> > > > >> >> > > >
>>> > > > >> >> > > >
>>> > > > >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>>> > > > >> >> > > >
>>> > > > >> >> > > > > What actually happened in brick4 (the node that
>>> refused
>>> > > > >> >> > r910(master)'s
>>> > > > >> >> > > > > connection) is that it
>>> > > > >> >> > > > > " failed to remove existing native library file: /tmp/
>>> > > > >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
>>> > > > >> >> > > > >         at
>>> > > > >> >> > > > >
>>> > > > >> >> > >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> >
>>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
>>> > > > >> >> > > > > then
>>> > > > >> >> > > > > java.lang.NullPointerException
>>> > > > >> >> > > > >         at
>>> > > > >> >> > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>>> > > > >> >> > > > >
>>> > > > >> >> > > > > But I actually run in the Hama directory and
>>> > > > >> >> snappy-java-1.0.4.1.jar
>>> > > > >> >> > is
>>> > > > >> >> > > > > included in hama/lib dir.
>>> > > > >> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
>>> > > > >> CLASSPATH.
>>> > > > >> >> > > > > What happened?
>>> > > > >> >> > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>>> > > > >> >> > > > >
>>> > > > >> >> > > > > > Hi, all,
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > > > I newly set up a 3-node hama cluster following the
>>> > > > >> >> > > > HamaInstallationGuide.
>>> > > > >> >> > > > > > But I got some confusing errors when running the pi
>>> > > > examples,
>>> > > > >> >> the
>>> > > > >> >> > > > tasklog
>>> > > > >> >> > > > > > is as follows:
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper:
>>> Initiating
>>> > > > client
>>> > > > >> >> > > > connection,
>>> > > > >> >> > > > > > connectString=brick4:
>>> > > > >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
>>> > > > >> >> > > > > >
>>> watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
>>> > > > >> >> > > > > > l@e61a35
>>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening
>>> > > socket
>>> > > > >> >> > > connection
>>> > > > >> >> > > > to
>>> > > > >> >> > > > > > server brick4/10.131.
>>> > > > >> >> > > > > > 201.14:21810
>>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl:
>>> > > Start
>>> > > > >> >> > connecting
>>> > > > >> >> > > > to
>>> > > > >> >> > > > > > Zookeeper! At r910.ppi
>>> > > > >> >> > > > > > /10.131.201.90:61001
>>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket
>>> > > > >> connection
>>> > > > >> >> > > > > established
>>> > > > >> >> > > > > > to brick4/10.131.201
>>> > > > >> >> > > > > > .14:21810, initiating session
>>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
>>> > > > >> >> establishment
>>> > > > >> >> > > > > > complete on server brick4/10
>>> > > > >> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c,
>>> > > > negotiated
>>> > > > >> >> > timeout
>>> > > > >> >> > > =
>>> > > > >> >> > > > > > 1200000
>>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
>>> Connecting
>>> > > to
>>> > > > >> >> > > brick4.r715/
>>> > > > >> >> > > > > > 10.131.201.14:61003
>>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
>>> > > > 0x0118278a]
>>> > > > >> >> OPEN
>>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
>>> > > > >> >> > > > > >         at
>>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>>> > > > >> >> Method)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > >
>>> > > > >>
>>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
>>> > > > >> >> > > > > > tSocketPipelineSink.java:384)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
>>> > > > >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
>>> > > > >> >> > > > > > ketPipelineSink.java:276)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
>>> > > > 0x0118278a]
>>> > > > >> >> > CLOSED
>>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote
>>> > peer
>>> > > > >> >> > brick4.r715/
>>> > > > >> >> > > > > > 10.131.201.14:61003 closed
>>> > > > >> >> > > > > >  connection.
>>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running
>>> bsp
>>> > > > setup
>>> > > > >> and
>>> > > > >> >> > bsp
>>> > > > >> >> > > > > > function.
>>> > > > >> >> > > > > > java.io.IOException: Error connecting to
>>> brick4.r715/
>>> > > > >> >> > > > 10.131.201.14:61003
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > >
>>> > > > >>
>>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> >
>>> > > >
>>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
>>> > > > >> >> > > > > > ava:83)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> >
>>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>>> > > > >> >> > > > > >         at
>>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>> > > > >> >> > > > > >         at
>>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > >
>>> > > > >> >> >
>>> > > > >>
>>> > >
>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
>>> > refused
>>> > > > >> >> > > > > >         at
>>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>>> > > > >> >> Method)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > >
>>> > > > >>
>>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>>> > > > >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver:
>>> Unexpected
>>> > > > >> >> exception
>>> > > > >> >> > > from
>>> > > > >> >> > > > > > downstream.
>>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
>>> > > > >> >> > > > > >         at
>>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>>> > > > >> >> Method)
>>> > > > >> >> > > > > >  at
>>> > > > >> >> > > > >
>>> > > > >> >> >
>>> > > > >>
>>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
>>> > > > >> >> > > 0x139b47286bf000c
>>> > > > >> >> > > > > > closed
>>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn:
>>> > EventThread
>>> > > > shut
>>> > > > >> >> down
>>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down
>>> ping
>>> > > > >> service.
>>> > > > >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error
>>> running
>>> > > > child
>>> > > > >> >> > > > > > java.io.IOException: Error connecting to
>>> brick4.r715/
>>> > > > >> >> > > > 10.131.201.14:61003
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > >
>>> > > > >>
>>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> >
>>> > > >
>>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> >
>>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>>> > > > >> >> > > > > >         at
>>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>> > > > >> >> > > > > >         at
>>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > >
>>> > > > >> >> >
>>> > > > >>
>>> > >
>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
>>> > refused
>>> > > > >> >> > > > > >         at
>>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>>> > > > >> >> Method)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > >
>>> > > > >>
>>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>>> > > > >> >> > > > > > java.io.IOException: Error connecting to
>>> brick4.r715/
>>> > > > >> >> > > > 10.131.201.14:61003
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > >
>>> > > > >>
>>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> >
>>> > > >
>>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> >
>>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>>> > > > >> >> > > > > >         at
>>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>> > > > >> >> > > > > >         at
>>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > >
>>> > > > >> >> >
>>> > > > >>
>>> > >
>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
>>> > refused
>>> > > > >> >> > > > > >         at
>>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>>> > > > >> >> Method)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > >
>>> > > > >>
>>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>>> > > > >> >> > > > > >         at
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> > > > >> >> > > > > > .....
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > > > Can anybody help? I am really desparate...
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > > > Thanks in advance,
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > > > Sandy
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > > >
>>> > > > >> >> > > > >
>>> > > > >> >> > > >
>>> > > > >> >> > >
>>> > > > >> >> >
>>> > > > >> >>
>>> > > > >> >
>>> > > > >> >
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
Sorry about the incomplete information.
I have changed the default filesystem to "file:///" instead of "local"
But the error still occurred:
   Error reading task
outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
Anything special to do if the default filesystem is set to lcoal?

2012/9/12 Sandy Ding <sa...@gmail.com>

> Yet another problem occurred...
> The Installation Guide says the value for fs.default.name is either local
> or host:port for HDFS.
> But when I set it to "local" to run pagerank, the system kept complain :
> "local is  "
>  Error reading task
> outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout
>
>
> 2012/9/12 Thomas Jungblut <th...@gmail.com>
>
>> Hey,
>>
>> it now takes plain text as input. Sorry the wikipedia page is deprecated,
>> will update it then.
>>
>> The format should be:
>>
>> VERTEX_ID\t(n-tab separated VERTEX_IDs) \n
>> E.g:
>> 1\t2\t3\t4 // makes 1, adjacent to 2, 3 and 4
>> 2\t3\t1
>>
>> 2012/9/12 Sandy Ding <sa...@gmail.com>
>>
>> > Hi,
>> >
>> > The pagerank-text2seq seems to have been removed from the
>> 0.5.0-examples?
>> > I got the error:
>> >
>> > Unknown program 'pagerank-text2seq' chosen.
>> >
>> > So how should I run the pagerank example now?
>> >
>> >
>> > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> >
>> > > On the machine where you submit the job, you can delete the
>> > > "hama.messenger.compression.class" property in your default
>> > configuration.
>> > > This should null out the compressor and make it run without
>> compression.
>> > >
>> > >
>> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > >
>> > > > Plus, how to remove the snappy compression form the configuration?
>> > > >
>> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > > >
>> > > > > It actually failed on another node brick5 too.
>> > > > > I do have the write permission of /tmp dir.
>> > > > > But the most weird thing is:
>> > > > >  the snappy..so file is generated every time I run the example,
>> but
>> > its
>> > > > > owner is another user who has once run hama on the same machine,
>> even
>> > > > when
>> > > > > I manually rm and re-generate the file! And I don't have the
>> > permission
>> > > > to
>> > > > > rm it!
>> > > > >
>> > > > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> > > > >
>> > > > >> Oh, or just remove the snappy compression from the configuration
>> ;)
>> > > > >>
>> > > > >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> > > > >>
>> > > > >> > Can you give him permission to delete in /tmp/?
>> > > > >> > Why is it not failing on the other hosts?
>> > > > >> > Otherwise you have to manually add the snappy library to the
>> lib
>> > > path
>> > > > of
>> > > > >> > the task.
>> > > > >> >
>> > > > >> >
>> > > > >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > > > >> >
>> > > > >> >> the IOException stack:
>> > > > >> >>
>> > > > >> >> java.io.IOException: failed to remove existing native library
>> > file:
>> > > > >> >> /tmp/snappy-1.0.4.1-libsna
>> > > > >> >> ppyjava.so
>> > > > >> >>         at
>> > > > >> >>
>> > > > >>
>> > >
>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
>> > > > >> >>         at
>> > > > >> >>
>> > > >
>> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
>> > > > >> >>         at
>> > > > >> >>
>> > > >
>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
>> > > > >> >>         at
>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>> > > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>> > > > >> >>         at
>> > > > >> >>
>> > > >
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>> > > > >> >>         at
>> > > > >> >>
>> > > >
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
>> > > > >> >> or.java:43)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
>> > > > >> >> gerImpl.java:135)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
>> > > > >> >> java:79)
>> > > > >> >>         at
>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >>
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > >> >>         at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >>
>> > >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > >> >> java.lang.reflect.InvocationTargetException
>> > > > >> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> > > > Method)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
>> > > > >> >> 5)
>> > > > >> >>         at java.lang.reflect.Method.invoke(Method.java:597)
>> > > > >> >>         at
>> > > > >> >>
>> > > >
>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
>> > > > >> >>         at
>> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>> > > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>> > > > >> >>         at
>> > > > >> >>
>> > > >
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>> > > > >> >>         at
>> > > > >> >>
>> > > >
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>> > > > >> >>         at
>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >>
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > >> >>         at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >>
>> > >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
>> > > > >> >> java.library.path
>> > > > >> >>         at
>> > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
>> > > > >> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
>> > > > >> >>         at java.lang.System.loadLibrary(System.java:1028)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
>> > > > >> >>         ... 17 more
>> > > > >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup
>> and
>> > > bsp
>> > > > >> >> function.
>> > > > >> >> java.lang.NullPointerException
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>> > > > >> >>         at
>> > > > >> >>
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>> > > > >> >>         at
>> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > >> >>
>> > > > >> >> the user that I currently run hama doesn't have  permissions
>> to
>> > > > remove
>> > > > >> >> /tmp/.  files.
>> > > > >> >> but I cannot run hama using sudo.
>> > > > >> >> doesn't what to do...
>> > > > >> >>
>> > > > >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> > > > >> >>
>> > > > >> >> > If you don't have additional stack information it is
>> difficult
>> > to
>> > > > >> guess.
>> > > > >> >> > According to
>> > > > >> >> >
>> > > > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
>> > > > >> >> > it should throw an IOException that contains the underlying
>> > > error.
>> > > > If
>> > > > >> >> the
>> > > > >> >> > shared object can't be deleted, I would try to start by
>> > checking
>> > > > >> rights
>> > > > >> >> or
>> > > > >> >> > what on the other machines is not like on the machine that
>> > > failed.
>> > > > >> >> >
>> > > > >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > > > >> >> >
>> > > > >> >> > > Any possible reasons for that?
>> > > > >> >> > >
>> > > > >> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> > > > >> >> > >
>> > > > >> >> > > > Hi,
>> > > > >> >> > > >
>> > > > >> >> > > > that is actually a good question.
>> > > > >> >> > > > According to the exception and the code [1] it was
>> unable
>> > to
>> > > > >> delete
>> > > > >> >> the
>> > > > >> >> > > > file.
>> > > > >> >> > > > Actually this could have various reasons, what operating
>> > > system
>> > > > >> are
>> > > > >> >> you
>> > > > >> >> > > > running?
>> > > > >> >> > > >
>> > > > >> >> > > > [1]
>> > > > >> >> > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
>> > > > >> >> > > >
>> > > > >> >> > > >
>> > > > >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > > > >> >> > > >
>> > > > >> >> > > > > What actually happened in brick4 (the node that
>> refused
>> > > > >> >> > r910(master)'s
>> > > > >> >> > > > > connection) is that it
>> > > > >> >> > > > > " failed to remove existing native library file: /tmp/
>> > > > >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
>> > > > >> >> > > > >         at
>> > > > >> >> > > > >
>> > > > >> >> > >
>> > > > >> >>
>> > > > >>
>> > > >
>> >
>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
>> > > > >> >> > > > > then
>> > > > >> >> > > > > java.lang.NullPointerException
>> > > > >> >> > > > >         at
>> > > > >> >> > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>> > > > >> >> > > > >
>> > > > >> >> > > > > But I actually run in the Hama directory and
>> > > > >> >> snappy-java-1.0.4.1.jar
>> > > > >> >> > is
>> > > > >> >> > > > > included in hama/lib dir.
>> > > > >> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
>> > > > >> CLASSPATH.
>> > > > >> >> > > > > What happened?
>> > > > >> >> > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > > > >> >> > > > >
>> > > > >> >> > > > > > Hi, all,
>> > > > >> >> > > > > >
>> > > > >> >> > > > > > I newly set up a 3-node hama cluster following the
>> > > > >> >> > > > HamaInstallationGuide.
>> > > > >> >> > > > > > But I got some confusing errors when running the pi
>> > > > examples,
>> > > > >> >> the
>> > > > >> >> > > > tasklog
>> > > > >> >> > > > > > is as follows:
>> > > > >> >> > > > > >
>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper:
>> Initiating
>> > > > client
>> > > > >> >> > > > connection,
>> > > > >> >> > > > > > connectString=brick4:
>> > > > >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
>> > > > >> >> > > > > >
>> watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
>> > > > >> >> > > > > > l@e61a35
>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening
>> > > socket
>> > > > >> >> > > connection
>> > > > >> >> > > > to
>> > > > >> >> > > > > > server brick4/10.131.
>> > > > >> >> > > > > > 201.14:21810
>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl:
>> > > Start
>> > > > >> >> > connecting
>> > > > >> >> > > > to
>> > > > >> >> > > > > > Zookeeper! At r910.ppi
>> > > > >> >> > > > > > /10.131.201.90:61001
>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket
>> > > > >> connection
>> > > > >> >> > > > > established
>> > > > >> >> > > > > > to brick4/10.131.201
>> > > > >> >> > > > > > .14:21810, initiating session
>> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
>> > > > >> >> establishment
>> > > > >> >> > > > > > complete on server brick4/10
>> > > > >> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c,
>> > > > negotiated
>> > > > >> >> > timeout
>> > > > >> >> > > =
>> > > > >> >> > > > > > 1200000
>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
>> Connecting
>> > > to
>> > > > >> >> > > brick4.r715/
>> > > > >> >> > > > > > 10.131.201.14:61003
>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
>> > > > 0x0118278a]
>> > > > >> >> OPEN
>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
>> > > > >> >> > > > > >         at
>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> > > > >> >> Method)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > >
>> > > > >>
>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
>> > > > >> >> > > > > > tSocketPipelineSink.java:384)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
>> > > > >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
>> > > > >> >> > > > > > ketPipelineSink.java:276)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
>> > > > 0x0118278a]
>> > > > >> >> > CLOSED
>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote
>> > peer
>> > > > >> >> > brick4.r715/
>> > > > >> >> > > > > > 10.131.201.14:61003 closed
>> > > > >> >> > > > > >  connection.
>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running
>> bsp
>> > > > setup
>> > > > >> and
>> > > > >> >> > bsp
>> > > > >> >> > > > > > function.
>> > > > >> >> > > > > > java.io.IOException: Error connecting to
>> brick4.r715/
>> > > > >> >> > > > 10.131.201.14:61003
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > >
>> > > > >>
>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> >
>> > > >
>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
>> > > > >> >> > > > > > ava:83)
>> > > > >> >> > > > > >         at
>> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > >> >> > > > > >         at
>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > >> >> > > > > >         at
>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > >
>> > > > >> >> >
>> > > > >>
>> > >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
>> > refused
>> > > > >> >> > > > > >         at
>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> > > > >> >> Method)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > >
>> > > > >>
>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver:
>> Unexpected
>> > > > >> >> exception
>> > > > >> >> > > from
>> > > > >> >> > > > > > downstream.
>> > > > >> >> > > > > > java.net.ConnectException: Connection refused
>> > > > >> >> > > > > >         at
>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> > > > >> >> Method)
>> > > > >> >> > > > > >  at
>> > > > >> >> > > > >
>> > > > >> >> >
>> > > > >>
>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
>> > > > >> >> > > 0x139b47286bf000c
>> > > > >> >> > > > > > closed
>> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn:
>> > EventThread
>> > > > shut
>> > > > >> >> down
>> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down
>> ping
>> > > > >> service.
>> > > > >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error
>> running
>> > > > child
>> > > > >> >> > > > > > java.io.IOException: Error connecting to
>> brick4.r715/
>> > > > >> >> > > > 10.131.201.14:61003
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > >
>> > > > >>
>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> >
>> > > >
>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>> > > > >> >> > > > > >         at
>> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > >> >> > > > > >         at
>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > >> >> > > > > >         at
>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > >
>> > > > >> >> >
>> > > > >>
>> > >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
>> > refused
>> > > > >> >> > > > > >         at
>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> > > > >> >> Method)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > >
>> > > > >>
>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > >> >> > > > > > java.io.IOException: Error connecting to
>> brick4.r715/
>> > > > >> >> > > > 10.131.201.14:61003
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > >
>> > > > >>
>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> >
>> > > >
>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>> > > > >> >> > > > > >         at
>> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > >> >> > > > > >         at
>> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > >> >> > > > > >         at
>> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > >
>> > > > >> >> >
>> > > > >>
>> > >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
>> > refused
>> > > > >> >> > > > > >         at
>> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> > > > >> >> Method)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > >
>> > > > >>
>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > >> >> > > > > >         at
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >>
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > >> >> > > > > > .....
>> > > > >> >> > > > > >
>> > > > >> >> > > > > > Can anybody help? I am really desparate...
>> > > > >> >> > > > > >
>> > > > >> >> > > > > > Thanks in advance,
>> > > > >> >> > > > > >
>> > > > >> >> > > > > > Sandy
>> > > > >> >> > > > > >
>> > > > >> >> > > > > >
>> > > > >> >> > > > >
>> > > > >> >> > > >
>> > > > >> >> > >
>> > > > >> >> >
>> > > > >> >>
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
Yet another problem occurred...
The Installation Guide says the value for fs.default.name is either local
or host:port for HDFS.
But when I set it to "local" to run pagerank, the system kept complain :
"local is  "
 Error reading task
outputhttp://brick5.r715:40015/tasklog?plaintext=true&taskid=attempt_201209121950_0001_000005_0&filter=stdout

2012/9/12 Thomas Jungblut <th...@gmail.com>

> Hey,
>
> it now takes plain text as input. Sorry the wikipedia page is deprecated,
> will update it then.
>
> The format should be:
>
> VERTEX_ID\t(n-tab separated VERTEX_IDs) \n
> E.g:
> 1\t2\t3\t4 // makes 1, adjacent to 2, 3 and 4
> 2\t3\t1
>
> 2012/9/12 Sandy Ding <sa...@gmail.com>
>
> > Hi,
> >
> > The pagerank-text2seq seems to have been removed from the 0.5.0-examples?
> > I got the error:
> >
> > Unknown program 'pagerank-text2seq' chosen.
> >
> > So how should I run the pagerank example now?
> >
> >
> > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >
> > > On the machine where you submit the job, you can delete the
> > > "hama.messenger.compression.class" property in your default
> > configuration.
> > > This should null out the compressor and make it run without
> compression.
> > >
> > >
> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >
> > > > Plus, how to remove the snappy compression form the configuration?
> > > >
> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > >
> > > > > It actually failed on another node brick5 too.
> > > > > I do have the write permission of /tmp dir.
> > > > > But the most weird thing is:
> > > > >  the snappy..so file is generated every time I run the example, but
> > its
> > > > > owner is another user who has once run hama on the same machine,
> even
> > > > when
> > > > > I manually rm and re-generate the file! And I don't have the
> > permission
> > > > to
> > > > > rm it!
> > > > >
> > > > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > > >
> > > > >> Oh, or just remove the snappy compression from the configuration
> ;)
> > > > >>
> > > > >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > > >>
> > > > >> > Can you give him permission to delete in /tmp/?
> > > > >> > Why is it not failing on the other hosts?
> > > > >> > Otherwise you have to manually add the snappy library to the lib
> > > path
> > > > of
> > > > >> > the task.
> > > > >> >
> > > > >> >
> > > > >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > > >> >
> > > > >> >> the IOException stack:
> > > > >> >>
> > > > >> >> java.io.IOException: failed to remove existing native library
> > file:
> > > > >> >> /tmp/snappy-1.0.4.1-libsna
> > > > >> >> ppyjava.so
> > > > >> >>         at
> > > > >> >>
> > > > >>
> > >
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
> > > > >> >>         at
> > > > >> >>
> > > >
> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
> > > > >> >>         at
> > > > >> >>
> > > >
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
> > > > >> >>         at
> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > > > >> >>         at
> > > > >> >>
> > > >
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > > > >> >>         at
> > > > >> >>
> > > >
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> > > > >> >> or.java:43)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> > > > >> >> gerImpl.java:135)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> > > > >> >> java:79)
> > > > >> >>         at
> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >> >>         at
> > > > >> >>
> > > > >>
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >> >>         at
> > > > >> >>
> > > > >>
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > >> >> java.lang.reflect.InvocationTargetException
> > > > >> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > > Method)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> > > > >> >> 5)
> > > > >> >>         at java.lang.reflect.Method.invoke(Method.java:597)
> > > > >> >>         at
> > > > >> >>
> > > >
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> > > > >> >>         at
> > > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > > > >> >>         at
> > > > >> >>
> > > >
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > > > >> >>         at
> > > > >> >>
> > > >
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > > > >> >>         at
> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >> >>         at
> > > > >> >>
> > > > >>
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >> >>         at
> > > > >> >>
> > > > >>
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
> > > > >> >> java.library.path
> > > > >> >>         at
> > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> > > > >> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
> > > > >> >>         at java.lang.System.loadLibrary(System.java:1028)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> > > > >> >>         ... 17 more
> > > > >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup
> and
> > > bsp
> > > > >> >> function.
> > > > >> >> java.lang.NullPointerException
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > > > >> >>         at
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > > > >> >>         at
> > > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >> >>
> > > > >> >> the user that I currently run hama doesn't have  permissions to
> > > > remove
> > > > >> >> /tmp/.  files.
> > > > >> >> but I cannot run hama using sudo.
> > > > >> >> doesn't what to do...
> > > > >> >>
> > > > >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > > >> >>
> > > > >> >> > If you don't have additional stack information it is
> difficult
> > to
> > > > >> guess.
> > > > >> >> > According to
> > > > >> >> >
> > > > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> > > > >> >> > it should throw an IOException that contains the underlying
> > > error.
> > > > If
> > > > >> >> the
> > > > >> >> > shared object can't be deleted, I would try to start by
> > checking
> > > > >> rights
> > > > >> >> or
> > > > >> >> > what on the other machines is not like on the machine that
> > > failed.
> > > > >> >> >
> > > > >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > > >> >> >
> > > > >> >> > > Any possible reasons for that?
> > > > >> >> > >
> > > > >> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > > >> >> > >
> > > > >> >> > > > Hi,
> > > > >> >> > > >
> > > > >> >> > > > that is actually a good question.
> > > > >> >> > > > According to the exception and the code [1] it was unable
> > to
> > > > >> delete
> > > > >> >> the
> > > > >> >> > > > file.
> > > > >> >> > > > Actually this could have various reasons, what operating
> > > system
> > > > >> are
> > > > >> >> you
> > > > >> >> > > > running?
> > > > >> >> > > >
> > > > >> >> > > > [1]
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > > >> >> > > >
> > > > >> >> > > > > What actually happened in brick4 (the node that refused
> > > > >> >> > r910(master)'s
> > > > >> >> > > > > connection) is that it
> > > > >> >> > > > > " failed to remove existing native library file: /tmp/
> > > > >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
> > > > >> >> > > > >         at
> > > > >> >> > > > >
> > > > >> >> > >
> > > > >> >>
> > > > >>
> > > >
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > > > >> >> > > > > then
> > > > >> >> > > > > java.lang.NullPointerException
> > > > >> >> > > > >         at
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > > > >> >> > > > >
> > > > >> >> > > > > But I actually run in the Hama directory and
> > > > >> >> snappy-java-1.0.4.1.jar
> > > > >> >> > is
> > > > >> >> > > > > included in hama/lib dir.
> > > > >> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
> > > > >> CLASSPATH.
> > > > >> >> > > > > What happened?
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > > >> >> > > > >
> > > > >> >> > > > > > Hi, all,
> > > > >> >> > > > > >
> > > > >> >> > > > > > I newly set up a 3-node hama cluster following the
> > > > >> >> > > > HamaInstallationGuide.
> > > > >> >> > > > > > But I got some confusing errors when running the pi
> > > > examples,
> > > > >> >> the
> > > > >> >> > > > tasklog
> > > > >> >> > > > > > is as follows:
> > > > >> >> > > > > >
> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper:
> Initiating
> > > > client
> > > > >> >> > > > connection,
> > > > >> >> > > > > > connectString=brick4:
> > > > >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
> > > > >> >> > > > > >
> watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > > > >> >> > > > > > l@e61a35
> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening
> > > socket
> > > > >> >> > > connection
> > > > >> >> > > > to
> > > > >> >> > > > > > server brick4/10.131.
> > > > >> >> > > > > > 201.14:21810
> > > > >> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl:
> > > Start
> > > > >> >> > connecting
> > > > >> >> > > > to
> > > > >> >> > > > > > Zookeeper! At r910.ppi
> > > > >> >> > > > > > /10.131.201.90:61001
> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket
> > > > >> connection
> > > > >> >> > > > > established
> > > > >> >> > > > > > to brick4/10.131.201
> > > > >> >> > > > > > .14:21810, initiating session
> > > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
> > > > >> >> establishment
> > > > >> >> > > > > > complete on server brick4/10
> > > > >> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c,
> > > > negotiated
> > > > >> >> > timeout
> > > > >> >> > > =
> > > > >> >> > > > > > 1200000
> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver:
> Connecting
> > > to
> > > > >> >> > > brick4.r715/
> > > > >> >> > > > > > 10.131.201.14:61003
> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> > > > 0x0118278a]
> > > > >> >> OPEN
> > > > >> >> > > > > > java.net.ConnectException: Connection refused
> > > > >> >> > > > > >         at
> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > > >> >> Method)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > > > >> >> > > > > > tSocketPipelineSink.java:384)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > > > >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > > > >> >> > > > > > ketPipelineSink.java:276)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> > > > 0x0118278a]
> > > > >> >> > CLOSED
> > > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote
> > peer
> > > > >> >> > brick4.r715/
> > > > >> >> > > > > > 10.131.201.14:61003 closed
> > > > >> >> > > > > >  connection.
> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running
> bsp
> > > > setup
> > > > >> and
> > > > >> >> > bsp
> > > > >> >> > > > > > function.
> > > > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > > >> >> > > > 10.131.201.14:61003
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> >
> > > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > > > >> >> > > > > > ava:83)
> > > > >> >> > > > > >         at
> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >> >> > > > > >         at
> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >> >> > > > > >         at
> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > >
> > > > >> >> >
> > > > >>
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> > refused
> > > > >> >> > > > > >         at
> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > > >> >> Method)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver:
> Unexpected
> > > > >> >> exception
> > > > >> >> > > from
> > > > >> >> > > > > > downstream.
> > > > >> >> > > > > > java.net.ConnectException: Connection refused
> > > > >> >> > > > > >         at
> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > > >> >> Method)
> > > > >> >> > > > > >  at
> > > > >> >> > > > >
> > > > >> >> >
> > > > >>
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> > > > >> >> > > 0x139b47286bf000c
> > > > >> >> > > > > > closed
> > > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn:
> > EventThread
> > > > shut
> > > > >> >> down
> > > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down
> ping
> > > > >> service.
> > > > >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error
> running
> > > > child
> > > > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > > >> >> > > > 10.131.201.14:61003
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> >
> > > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > > >> >> > > > > >         at
> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >> >> > > > > >         at
> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >> >> > > > > >         at
> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > >
> > > > >> >> >
> > > > >>
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> > refused
> > > > >> >> > > > > >         at
> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > > >> >> Method)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > > >> >> > > > 10.131.201.14:61003
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> >
> > > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > > >> >> > > > > >         at
> > > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >> >> > > > > >         at
> > > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >> >> > > > > >         at
> > > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > >
> > > > >> >> >
> > > > >>
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> > refused
> > > > >> >> > > > > >         at
> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > > >> >> Method)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >> >> > > > > >         at
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >> >> > > > > > .....
> > > > >> >> > > > > >
> > > > >> >> > > > > > Can anybody help? I am really desparate...
> > > > >> >> > > > > >
> > > > >> >> > > > > > Thanks in advance,
> > > > >> >> > > > > >
> > > > >> >> > > > > > Sandy
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Thomas Jungblut <th...@gmail.com>.
Hey,

it now takes plain text as input. Sorry the wikipedia page is deprecated,
will update it then.

The format should be:

VERTEX_ID\t(n-tab separated VERTEX_IDs) \n
E.g:
1\t2\t3\t4 // makes 1, adjacent to 2, 3 and 4
2\t3\t1

2012/9/12 Sandy Ding <sa...@gmail.com>

> Hi,
>
> The pagerank-text2seq seems to have been removed from the 0.5.0-examples?
> I got the error:
>
> Unknown program 'pagerank-text2seq' chosen.
>
> So how should I run the pagerank example now?
>
>
> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>
> > On the machine where you submit the job, you can delete the
> > "hama.messenger.compression.class" property in your default
> configuration.
> > This should null out the compressor and make it run without compression.
> >
> >
> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >
> > > Plus, how to remove the snappy compression form the configuration?
> > >
> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >
> > > > It actually failed on another node brick5 too.
> > > > I do have the write permission of /tmp dir.
> > > > But the most weird thing is:
> > > >  the snappy..so file is generated every time I run the example, but
> its
> > > > owner is another user who has once run hama on the same machine, even
> > > when
> > > > I manually rm and re-generate the file! And I don't have the
> permission
> > > to
> > > > rm it!
> > > >
> > > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > >
> > > >> Oh, or just remove the snappy compression from the configuration ;)
> > > >>
> > > >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > >>
> > > >> > Can you give him permission to delete in /tmp/?
> > > >> > Why is it not failing on the other hosts?
> > > >> > Otherwise you have to manually add the snappy library to the lib
> > path
> > > of
> > > >> > the task.
> > > >> >
> > > >> >
> > > >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > >> >
> > > >> >> the IOException stack:
> > > >> >>
> > > >> >> java.io.IOException: failed to remove existing native library
> file:
> > > >> >> /tmp/snappy-1.0.4.1-libsna
> > > >> >> ppyjava.so
> > > >> >>         at
> > > >> >>
> > > >>
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
> > > >> >>         at
> > > >> >>
> > > org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
> > > >> >>         at
> > > >> >>
> > > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
> > > >> >>         at
> > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > > >> >>         at
> > > >> >>
> > > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > > >> >>         at
> > > >> >>
> > > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> > > >> >> or.java:43)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> > > >> >> gerImpl.java:135)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> > > >> >> java:79)
> > > >> >>         at
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >> >>         at
> > > >> >>
> > > >>
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >> >>         at
> > > >> >>
> > > >>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > >> >> java.lang.reflect.InvocationTargetException
> > > >> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> > > >> >> 5)
> > > >> >>         at java.lang.reflect.Method.invoke(Method.java:597)
> > > >> >>         at
> > > >> >>
> > > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> > > >> >>         at
> > org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > > >> >>         at
> > > >> >>
> > > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > > >> >>         at
> > > >> >>
> > > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > > >> >>         at
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >> >>         at
> > > >> >>
> > > >>
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >> >>         at
> > > >> >>
> > > >>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
> > > >> >> java.library.path
> > > >> >>         at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> > > >> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
> > > >> >>         at java.lang.System.loadLibrary(System.java:1028)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> > > >> >>         ... 17 more
> > > >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and
> > bsp
> > > >> >> function.
> > > >> >> java.lang.NullPointerException
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > > >> >>         at
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > > >> >>         at
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >> >>
> > > >> >> the user that I currently run hama doesn't have  permissions to
> > > remove
> > > >> >> /tmp/.  files.
> > > >> >> but I cannot run hama using sudo.
> > > >> >> doesn't what to do...
> > > >> >>
> > > >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > >> >>
> > > >> >> > If you don't have additional stack information it is difficult
> to
> > > >> guess.
> > > >> >> > According to
> > > >> >> >
> > > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> > > >> >> > it should throw an IOException that contains the underlying
> > error.
> > > If
> > > >> >> the
> > > >> >> > shared object can't be deleted, I would try to start by
> checking
> > > >> rights
> > > >> >> or
> > > >> >> > what on the other machines is not like on the machine that
> > failed.
> > > >> >> >
> > > >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > >> >> >
> > > >> >> > > Any possible reasons for that?
> > > >> >> > >
> > > >> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > > >> >> > >
> > > >> >> > > > Hi,
> > > >> >> > > >
> > > >> >> > > > that is actually a good question.
> > > >> >> > > > According to the exception and the code [1] it was unable
> to
> > > >> delete
> > > >> >> the
> > > >> >> > > > file.
> > > >> >> > > > Actually this could have various reasons, what operating
> > system
> > > >> are
> > > >> >> you
> > > >> >> > > > running?
> > > >> >> > > >
> > > >> >> > > > [1]
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > >> >> > > >
> > > >> >> > > > > What actually happened in brick4 (the node that refused
> > > >> >> > r910(master)'s
> > > >> >> > > > > connection) is that it
> > > >> >> > > > > " failed to remove existing native library file: /tmp/
> > > >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
> > > >> >> > > > >         at
> > > >> >> > > > >
> > > >> >> > >
> > > >> >>
> > > >>
> > >
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > > >> >> > > > > then
> > > >> >> > > > > java.lang.NullPointerException
> > > >> >> > > > >         at
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > > >> >> > > > >
> > > >> >> > > > > But I actually run in the Hama directory and
> > > >> >> snappy-java-1.0.4.1.jar
> > > >> >> > is
> > > >> >> > > > > included in hama/lib dir.
> > > >> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
> > > >> CLASSPATH.
> > > >> >> > > > > What happened?
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > >> >> > > > >
> > > >> >> > > > > > Hi, all,
> > > >> >> > > > > >
> > > >> >> > > > > > I newly set up a 3-node hama cluster following the
> > > >> >> > > > HamaInstallationGuide.
> > > >> >> > > > > > But I got some confusing errors when running the pi
> > > examples,
> > > >> >> the
> > > >> >> > > > tasklog
> > > >> >> > > > > > is as follows:
> > > >> >> > > > > >
> > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating
> > > client
> > > >> >> > > > connection,
> > > >> >> > > > > > connectString=brick4:
> > > >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
> > > >> >> > > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > > >> >> > > > > > l@e61a35
> > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening
> > socket
> > > >> >> > > connection
> > > >> >> > > > to
> > > >> >> > > > > > server brick4/10.131.
> > > >> >> > > > > > 201.14:21810
> > > >> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl:
> > Start
> > > >> >> > connecting
> > > >> >> > > > to
> > > >> >> > > > > > Zookeeper! At r910.ppi
> > > >> >> > > > > > /10.131.201.90:61001
> > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket
> > > >> connection
> > > >> >> > > > > established
> > > >> >> > > > > > to brick4/10.131.201
> > > >> >> > > > > > .14:21810, initiating session
> > > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
> > > >> >> establishment
> > > >> >> > > > > > complete on server brick4/10
> > > >> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c,
> > > negotiated
> > > >> >> > timeout
> > > >> >> > > =
> > > >> >> > > > > > 1200000
> > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting
> > to
> > > >> >> > > brick4.r715/
> > > >> >> > > > > > 10.131.201.14:61003
> > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> > > 0x0118278a]
> > > >> >> OPEN
> > > >> >> > > > > > java.net.ConnectException: Connection refused
> > > >> >> > > > > >         at
> > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > >> >> Method)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > > >> >> > > > > > tSocketPipelineSink.java:384)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > > >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > > >> >> > > > > > ketPipelineSink.java:276)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> > > 0x0118278a]
> > > >> >> > CLOSED
> > > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote
> peer
> > > >> >> > brick4.r715/
> > > >> >> > > > > > 10.131.201.14:61003 closed
> > > >> >> > > > > >  connection.
> > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp
> > > setup
> > > >> and
> > > >> >> > bsp
> > > >> >> > > > > > function.
> > > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > >> >> > > > 10.131.201.14:61003
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> >
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > > >> >> > > > > > ava:83)
> > > >> >> > > > > >         at
> > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >> >> > > > > >         at
> > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >> >> > > > > >         at
> > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > >
> > > >> >> >
> > > >>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> refused
> > > >> >> > > > > >         at
> > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > >> >> Method)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected
> > > >> >> exception
> > > >> >> > > from
> > > >> >> > > > > > downstream.
> > > >> >> > > > > > java.net.ConnectException: Connection refused
> > > >> >> > > > > >         at
> > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > >> >> Method)
> > > >> >> > > > > >  at
> > > >> >> > > > >
> > > >> >> >
> > > >>
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> > > >> >> > > 0x139b47286bf000c
> > > >> >> > > > > > closed
> > > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn:
> EventThread
> > > shut
> > > >> >> down
> > > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping
> > > >> service.
> > > >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running
> > > child
> > > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > >> >> > > > 10.131.201.14:61003
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> >
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > >> >> > > > > >         at
> > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >> >> > > > > >         at
> > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >> >> > > > > >         at
> > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > >
> > > >> >> >
> > > >>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> refused
> > > >> >> > > > > >         at
> > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > >> >> Method)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > >> >> > > > 10.131.201.14:61003
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> >
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > >> >> > > > > >         at
> > > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >> >> > > > > >         at
> > > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >> >> > > > > >         at
> > > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > >
> > > >> >> >
> > > >>
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > >> >> > > > > > Caused by: java.net.ConnectException: Connection
> refused
> > > >> >> > > > > >         at
> > sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > > >> >> Method)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >> >> > > > > >         at
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >> >> > > > > > .....
> > > >> >> > > > > >
> > > >> >> > > > > > Can anybody help? I am really desparate...
> > > >> >> > > > > >
> > > >> >> > > > > > Thanks in advance,
> > > >> >> > > > > >
> > > >> >> > > > > > Sandy
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
Hi,

The pagerank-text2seq seems to have been removed from the 0.5.0-examples?
I got the error:

Unknown program 'pagerank-text2seq' chosen.

So how should I run the pagerank example now?


2012/9/11 Thomas Jungblut <th...@gmail.com>

> On the machine where you submit the job, you can delete the
> "hama.messenger.compression.class" property in your default configuration.
> This should null out the compressor and make it run without compression.
>
>
> 2012/9/11 Sandy Ding <sa...@gmail.com>
>
> > Plus, how to remove the snappy compression form the configuration?
> >
> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >
> > > It actually failed on another node brick5 too.
> > > I do have the write permission of /tmp dir.
> > > But the most weird thing is:
> > >  the snappy..so file is generated every time I run the example, but its
> > > owner is another user who has once run hama on the same machine, even
> > when
> > > I manually rm and re-generate the file! And I don't have the permission
> > to
> > > rm it!
> > >
> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >
> > >> Oh, or just remove the snappy compression from the configuration ;)
> > >>
> > >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >>
> > >> > Can you give him permission to delete in /tmp/?
> > >> > Why is it not failing on the other hosts?
> > >> > Otherwise you have to manually add the snappy library to the lib
> path
> > of
> > >> > the task.
> > >> >
> > >> >
> > >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >> >
> > >> >> the IOException stack:
> > >> >>
> > >> >> java.io.IOException: failed to remove existing native library file:
> > >> >> /tmp/snappy-1.0.4.1-libsna
> > >> >> ppyjava.so
> > >> >>         at
> > >> >>
> > >>
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
> > >> >>         at
> > >> >>
> > org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
> > >> >>         at
> > >> >>
> > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
> > >> >>         at
> org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > >> >>         at
> > >> >>
> > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > >> >>         at
> > >> >>
> > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> > >> >> or.java:43)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> > >> >> gerImpl.java:135)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> > >> >> java:79)
> > >> >>         at
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >> >>         at
> > >> >>
> > >>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >> >>         at
> > >> >>
> > >>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >> >> java.lang.reflect.InvocationTargetException
> > >> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> > >> >> 5)
> > >> >>         at java.lang.reflect.Method.invoke(Method.java:597)
> > >> >>         at
> > >> >>
> > org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> > >> >>         at
> org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> > >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> > >> >>         at
> > >> >>
> > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> > >> >>         at
> > >> >>
> > org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > >> >>         at
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >> >>         at
> > >> >>
> > >>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >> >>         at
> > >> >>
> > >>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
> > >> >> java.library.path
> > >> >>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> > >> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
> > >> >>         at java.lang.System.loadLibrary(System.java:1028)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> > >> >>         ... 17 more
> > >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and
> bsp
> > >> >> function.
> > >> >> java.lang.NullPointerException
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> > >> >>         at
> > >> >>
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> > >> >>         at
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >> >>
> > >> >> the user that I currently run hama doesn't have  permissions to
> > remove
> > >> >> /tmp/.  files.
> > >> >> but I cannot run hama using sudo.
> > >> >> doesn't what to do...
> > >> >>
> > >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >> >>
> > >> >> > If you don't have additional stack information it is difficult to
> > >> guess.
> > >> >> > According to
> > >> >> >
> > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> > >> >> > it should throw an IOException that contains the underlying
> error.
> > If
> > >> >> the
> > >> >> > shared object can't be deleted, I would try to start by checking
> > >> rights
> > >> >> or
> > >> >> > what on the other machines is not like on the machine that
> failed.
> > >> >> >
> > >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >> >> >
> > >> >> > > Any possible reasons for that?
> > >> >> > >
> > >> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >> >> > >
> > >> >> > > > Hi,
> > >> >> > > >
> > >> >> > > > that is actually a good question.
> > >> >> > > > According to the exception and the code [1] it was unable to
> > >> delete
> > >> >> the
> > >> >> > > > file.
> > >> >> > > > Actually this could have various reasons, what operating
> system
> > >> are
> > >> >> you
> > >> >> > > > running?
> > >> >> > > >
> > >> >> > > > [1]
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >> >> > > >
> > >> >> > > > > What actually happened in brick4 (the node that refused
> > >> >> > r910(master)'s
> > >> >> > > > > connection) is that it
> > >> >> > > > > " failed to remove existing native library file: /tmp/
> > >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
> > >> >> > > > >         at
> > >> >> > > > >
> > >> >> > >
> > >> >>
> > >>
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > >> >> > > > > then
> > >> >> > > > > java.lang.NullPointerException
> > >> >> > > > >         at
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > >> >> > > > >
> > >> >> > > > > But I actually run in the Hama directory and
> > >> >> snappy-java-1.0.4.1.jar
> > >> >> > is
> > >> >> > > > > included in hama/lib dir.
> > >> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
> > >> CLASSPATH.
> > >> >> > > > > What happened?
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >> >> > > > >
> > >> >> > > > > > Hi, all,
> > >> >> > > > > >
> > >> >> > > > > > I newly set up a 3-node hama cluster following the
> > >> >> > > > HamaInstallationGuide.
> > >> >> > > > > > But I got some confusing errors when running the pi
> > examples,
> > >> >> the
> > >> >> > > > tasklog
> > >> >> > > > > > is as follows:
> > >> >> > > > > >
> > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating
> > client
> > >> >> > > > connection,
> > >> >> > > > > > connectString=brick4:
> > >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
> > >> >> > > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > >> >> > > > > > l@e61a35
> > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening
> socket
> > >> >> > > connection
> > >> >> > > > to
> > >> >> > > > > > server brick4/10.131.
> > >> >> > > > > > 201.14:21810
> > >> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl:
> Start
> > >> >> > connecting
> > >> >> > > > to
> > >> >> > > > > > Zookeeper! At r910.ppi
> > >> >> > > > > > /10.131.201.90:61001
> > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket
> > >> connection
> > >> >> > > > > established
> > >> >> > > > > > to brick4/10.131.201
> > >> >> > > > > > .14:21810, initiating session
> > >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
> > >> >> establishment
> > >> >> > > > > > complete on server brick4/10
> > >> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c,
> > negotiated
> > >> >> > timeout
> > >> >> > > =
> > >> >> > > > > > 1200000
> > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting
> to
> > >> >> > > brick4.r715/
> > >> >> > > > > > 10.131.201.14:61003
> > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> > 0x0118278a]
> > >> >> OPEN
> > >> >> > > > > > java.net.ConnectException: Connection refused
> > >> >> > > > > >         at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >> >> Method)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > >
> > >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > >> >> > > > > > tSocketPipelineSink.java:384)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > >> >> > > > > > ketPipelineSink.java:276)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> > 0x0118278a]
> > >> >> > CLOSED
> > >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer
> > >> >> > brick4.r715/
> > >> >> > > > > > 10.131.201.14:61003 closed
> > >> >> > > > > >  connection.
> > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp
> > setup
> > >> and
> > >> >> > bsp
> > >> >> > > > > > function.
> > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > >> >> > > > 10.131.201.14:61003
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > >
> > >> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > >> >> > > > > > ava:83)
> > >> >> > > > > >         at
> > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > >
> > >> >> >
> > >> >>
> > >>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >> >> > > > > >         at
> > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >> >> > > > > >         at
> > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > >
> > >> >> >
> > >>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> > >> >> > > > > >         at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >> >> Method)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > >
> > >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected
> > >> >> exception
> > >> >> > > from
> > >> >> > > > > > downstream.
> > >> >> > > > > > java.net.ConnectException: Connection refused
> > >> >> > > > > >         at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >> >> Method)
> > >> >> > > > > >  at
> > >> >> > > > >
> > >> >> >
> > >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> > >> >> > > 0x139b47286bf000c
> > >> >> > > > > > closed
> > >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread
> > shut
> > >> >> down
> > >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping
> > >> service.
> > >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running
> > child
> > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > >> >> > > > 10.131.201.14:61003
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > >
> > >> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >> >> > > > > >         at
> > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > >
> > >> >> >
> > >> >>
> > >>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >> >> > > > > >         at
> > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >> >> > > > > >         at
> > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > >
> > >> >> >
> > >>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> > >> >> > > > > >         at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >> >> Method)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > >
> > >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > >> >> > > > 10.131.201.14:61003
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > >
> > >> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >> >> > > > > >         at
> > >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > >
> > >> >> >
> > >> >>
> > >>
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >> >> > > > > >         at
> > >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >> >> > > > > >         at
> > org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > >
> > >> >> >
> > >>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > >> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> > >> >> > > > > >         at
> sun.nio.ch.SocketChannelImpl.checkConnect(Native
> > >> >> Method)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > >
> > >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >> >> > > > > >         at
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> >> > > > > > .....
> > >> >> > > > > >
> > >> >> > > > > > Can anybody help? I am really desparate...
> > >> >> > > > > >
> > >> >> > > > > > Thanks in advance,
> > >> >> > > > > >
> > >> >> > > > > > Sandy
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> > >> >
> > >>
> > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Thomas Jungblut <th...@gmail.com>.
On the machine where you submit the job, you can delete the
"hama.messenger.compression.class" property in your default configuration.
This should null out the compressor and make it run without compression.


2012/9/11 Sandy Ding <sa...@gmail.com>

> Plus, how to remove the snappy compression form the configuration?
>
> 2012/9/11 Sandy Ding <sa...@gmail.com>
>
> > It actually failed on another node brick5 too.
> > I do have the write permission of /tmp dir.
> > But the most weird thing is:
> >  the snappy..so file is generated every time I run the example, but its
> > owner is another user who has once run hama on the same machine, even
> when
> > I manually rm and re-generate the file! And I don't have the permission
> to
> > rm it!
> >
> > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >
> >> Oh, or just remove the snappy compression from the configuration ;)
> >>
> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >>
> >> > Can you give him permission to delete in /tmp/?
> >> > Why is it not failing on the other hosts?
> >> > Otherwise you have to manually add the snappy library to the lib path
> of
> >> > the task.
> >> >
> >> >
> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >> >
> >> >> the IOException stack:
> >> >>
> >> >> java.io.IOException: failed to remove existing native library file:
> >> >> /tmp/snappy-1.0.4.1-libsna
> >> >> ppyjava.so
> >> >>         at
> >> >>
> >> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
> >> >>         at
> >> >>
> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
> >> >>         at
> >> >>
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
> >> >>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> >> >>         at
> >> >>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> >> >>         at
> >> >>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> >> >> or.java:43)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> >> >> gerImpl.java:135)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> >> >> java:79)
> >> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> >>         at
> >> >>
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> >>         at
> >> >>
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> >> java.lang.reflect.InvocationTargetException
> >> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> >> >>         at
> >> >>
> >> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> >>         at
> >> >>
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> >> >> 5)
> >> >>         at java.lang.reflect.Method.invoke(Method.java:597)
> >> >>         at
> >> >>
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> >> >>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> >> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> >> >>         at
> >> >>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> >> >>         at
> >> >>
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> >> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> >>         at
> >> >>
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> >>         at
> >> >>
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
> >> >> java.library.path
> >> >>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> >> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
> >> >>         at java.lang.System.loadLibrary(System.java:1028)
> >> >>         at
> >> >>
> >> >>
> >>
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> >> >>         ... 17 more
> >> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and bsp
> >> >> function.
> >> >> java.lang.NullPointerException
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> >> >>         at
> >> >>
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> >> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> >>
> >> >> the user that I currently run hama doesn't have  permissions to
> remove
> >> >> /tmp/.  files.
> >> >> but I cannot run hama using sudo.
> >> >> doesn't what to do...
> >> >>
> >> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >> >>
> >> >> > If you don't have additional stack information it is difficult to
> >> guess.
> >> >> > According to
> >> >> >
> http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> >> >> > it should throw an IOException that contains the underlying error.
> If
> >> >> the
> >> >> > shared object can't be deleted, I would try to start by checking
> >> rights
> >> >> or
> >> >> > what on the other machines is not like on the machine that failed.
> >> >> >
> >> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >> >> >
> >> >> > > Any possible reasons for that?
> >> >> > >
> >> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >> >> > >
> >> >> > > > Hi,
> >> >> > > >
> >> >> > > > that is actually a good question.
> >> >> > > > According to the exception and the code [1] it was unable to
> >> delete
> >> >> the
> >> >> > > > file.
> >> >> > > > Actually this could have various reasons, what operating system
> >> are
> >> >> you
> >> >> > > > running?
> >> >> > > >
> >> >> > > > [1]
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> >> >> > > >
> >> >> > > >
> >> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >> >> > > >
> >> >> > > > > What actually happened in brick4 (the node that refused
> >> >> > r910(master)'s
> >> >> > > > > connection) is that it
> >> >> > > > > " failed to remove existing native library file: /tmp/
> >> >> > > > > snappy-1.0.4.1-libsnappyjava.so
> >> >> > > > >         at
> >> >> > > > >
> >> >> > >
> >> >>
> >>
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> >> >> > > > > then
> >> >> > > > > java.lang.NullPointerException
> >> >> > > > >         at
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >> >> > > > >
> >> >> > > > > But I actually run in the Hama directory and
> >> >> snappy-java-1.0.4.1.jar
> >> >> > is
> >> >> > > > > included in hama/lib dir.
> >> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
> >> CLASSPATH.
> >> >> > > > > What happened?
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >> >> > > > >
> >> >> > > > > > Hi, all,
> >> >> > > > > >
> >> >> > > > > > I newly set up a 3-node hama cluster following the
> >> >> > > > HamaInstallationGuide.
> >> >> > > > > > But I got some confusing errors when running the pi
> examples,
> >> >> the
> >> >> > > > tasklog
> >> >> > > > > > is as follows:
> >> >> > > > > >
> >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating
> client
> >> >> > > > connection,
> >> >> > > > > > connectString=brick4:
> >> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
> >> >> > > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> >> >> > > > > > l@e61a35
> >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket
> >> >> > > connection
> >> >> > > > to
> >> >> > > > > > server brick4/10.131.
> >> >> > > > > > 201.14:21810
> >> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start
> >> >> > connecting
> >> >> > > > to
> >> >> > > > > > Zookeeper! At r910.ppi
> >> >> > > > > > /10.131.201.90:61001
> >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket
> >> connection
> >> >> > > > > established
> >> >> > > > > > to brick4/10.131.201
> >> >> > > > > > .14:21810, initiating session
> >> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
> >> >> establishment
> >> >> > > > > > complete on server brick4/10
> >> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c,
> negotiated
> >> >> > timeout
> >> >> > > =
> >> >> > > > > > 1200000
> >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to
> >> >> > > brick4.r715/
> >> >> > > > > > 10.131.201.14:61003
> >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> 0x0118278a]
> >> >> OPEN
> >> >> > > > > > java.net.ConnectException: Connection refused
> >> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> >> Method)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > >
> >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> >> >> > > > > > tSocketPipelineSink.java:384)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> >> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> >> >> > > > > > ketPipelineSink.java:276)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id:
> 0x0118278a]
> >> >> > CLOSED
> >> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer
> >> >> > brick4.r715/
> >> >> > > > > > 10.131.201.14:61003 closed
> >> >> > > > > >  connection.
> >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp
> setup
> >> and
> >> >> > bsp
> >> >> > > > > > function.
> >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> >> >> > > > 10.131.201.14:61003
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > >
> >> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> >> >> > > > > > ava:83)
> >> >> > > > > >         at
> >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > >
> >> >> >
> >> >>
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> >> > > > > >         at
> >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> >> > > > > >         at
> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > >
> >> >> >
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> >> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> >> Method)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > >
> >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected
> >> >> exception
> >> >> > > from
> >> >> > > > > > downstream.
> >> >> > > > > > java.net.ConnectException: Connection refused
> >> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> >> Method)
> >> >> > > > > >  at
> >> >> > > > >
> >> >> >
> >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> >> >> > > 0x139b47286bf000c
> >> >> > > > > > closed
> >> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread
> shut
> >> >> down
> >> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping
> >> service.
> >> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running
> child
> >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> >> >> > > > 10.131.201.14:61003
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > >
> >> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >> >> > > > > >         at
> >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > >
> >> >> >
> >> >>
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> >> > > > > >         at
> >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> >> > > > > >         at
> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > >
> >> >> >
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> >> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> >> Method)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > >
> >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> >> >> > > > 10.131.201.14:61003
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > >
> >> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >> >> > > > > >         at
> >> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > >
> >> >> >
> >> >>
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> >> > > > > >         at
> >> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> >> > > > > >         at
> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > >
> >> >> >
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> >> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> >> Method)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > >
> >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> >> > > > > >         at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >> > > > > > .....
> >> >> > > > > >
> >> >> > > > > > Can anybody help? I am really desparate...
> >> >> > > > > >
> >> >> > > > > > Thanks in advance,
> >> >> > > > > >
> >> >> > > > > > Sandy
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >> >
> >>
> >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
Plus, how to remove the snappy compression form the configuration?

2012/9/11 Sandy Ding <sa...@gmail.com>

> It actually failed on another node brick5 too.
> I do have the write permission of /tmp dir.
> But the most weird thing is:
>  the snappy..so file is generated every time I run the example, but its
> owner is another user who has once run hama on the same machine, even when
> I manually rm and re-generate the file! And I don't have the permission to
> rm it!
>
> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>
>> Oh, or just remove the snappy compression from the configuration ;)
>>
>> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>>
>> > Can you give him permission to delete in /tmp/?
>> > Why is it not failing on the other hosts?
>> > Otherwise you have to manually add the snappy library to the lib path of
>> > the task.
>> >
>> >
>> > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> >
>> >> the IOException stack:
>> >>
>> >> java.io.IOException: failed to remove existing native library file:
>> >> /tmp/snappy-1.0.4.1-libsna
>> >> ppyjava.so
>> >>         at
>> >>
>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
>> >>         at
>> >> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
>> >>         at
>> >> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
>> >>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>> >>         at
>> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>> >>         at
>> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
>> >> or.java:43)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
>> >> gerImpl.java:135)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
>> >> java:79)
>> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> >>         at
>> >>
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> >>         at
>> >>
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> >> java.lang.reflect.InvocationTargetException
>> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>         at
>> >>
>> >>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >>         at
>> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
>> >> 5)
>> >>         at java.lang.reflect.Method.invoke(Method.java:597)
>> >>         at
>> >> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
>> >>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>> >>         at
>> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>> >>         at
>> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> >>         at
>> >>
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> >>         at
>> >>
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
>> >> java.library.path
>> >>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
>> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
>> >>         at java.lang.System.loadLibrary(System.java:1028)
>> >>         at
>> >>
>> >>
>> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
>> >>         ... 17 more
>> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and bsp
>> >> function.
>> >> java.lang.NullPointerException
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>> >>         at
>> >>
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> >>
>> >> the user that I currently run hama doesn't have  permissions to remove
>> >> /tmp/.  files.
>> >> but I cannot run hama using sudo.
>> >> doesn't what to do...
>> >>
>> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> >>
>> >> > If you don't have additional stack information it is difficult to
>> guess.
>> >> > According to
>> >> > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
>> >> > it should throw an IOException that contains the underlying error. If
>> >> the
>> >> > shared object can't be deleted, I would try to start by checking
>> rights
>> >> or
>> >> > what on the other machines is not like on the machine that failed.
>> >> >
>> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> >> >
>> >> > > Any possible reasons for that?
>> >> > >
>> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> >> > >
>> >> > > > Hi,
>> >> > > >
>> >> > > > that is actually a good question.
>> >> > > > According to the exception and the code [1] it was unable to
>> delete
>> >> the
>> >> > > > file.
>> >> > > > Actually this could have various reasons, what operating system
>> are
>> >> you
>> >> > > > running?
>> >> > > >
>> >> > > > [1]
>> >> > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
>> >> > > >
>> >> > > >
>> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> >> > > >
>> >> > > > > What actually happened in brick4 (the node that refused
>> >> > r910(master)'s
>> >> > > > > connection) is that it
>> >> > > > > " failed to remove existing native library file: /tmp/
>> >> > > > > snappy-1.0.4.1-libsnappyjava.so
>> >> > > > >         at
>> >> > > > >
>> >> > >
>> >>
>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
>> >> > > > > then
>> >> > > > > java.lang.NullPointerException
>> >> > > > >         at
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>> >> > > > >
>> >> > > > > But I actually run in the Hama directory and
>> >> snappy-java-1.0.4.1.jar
>> >> > is
>> >> > > > > included in hama/lib dir.
>> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
>> CLASSPATH.
>> >> > > > > What happened?
>> >> > > > >
>> >> > > > >
>> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> >> > > > >
>> >> > > > > > Hi, all,
>> >> > > > > >
>> >> > > > > > I newly set up a 3-node hama cluster following the
>> >> > > > HamaInstallationGuide.
>> >> > > > > > But I got some confusing errors when running the pi examples,
>> >> the
>> >> > > > tasklog
>> >> > > > > > is as follows:
>> >> > > > > >
>> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
>> >> > > > connection,
>> >> > > > > > connectString=brick4:
>> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
>> >> > > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
>> >> > > > > > l@e61a35
>> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket
>> >> > > connection
>> >> > > > to
>> >> > > > > > server brick4/10.131.
>> >> > > > > > 201.14:21810
>> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start
>> >> > connecting
>> >> > > > to
>> >> > > > > > Zookeeper! At r910.ppi
>> >> > > > > > /10.131.201.90:61001
>> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket
>> connection
>> >> > > > > established
>> >> > > > > > to brick4/10.131.201
>> >> > > > > > .14:21810, initiating session
>> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
>> >> establishment
>> >> > > > > > complete on server brick4/10
>> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated
>> >> > timeout
>> >> > > =
>> >> > > > > > 1200000
>> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to
>> >> > > brick4.r715/
>> >> > > > > > 10.131.201.14:61003
>> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
>> >> OPEN
>> >> > > > > > java.net.ConnectException: Connection refused
>> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> >> Method)
>> >> > > > > >         at
>> >> > > > > >
>> >> > >
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
>> >> > > > > > tSocketPipelineSink.java:384)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
>> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
>> >> > > > > > ketPipelineSink.java:276)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
>> >> > CLOSED
>> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer
>> >> > brick4.r715/
>> >> > > > > > 10.131.201.14:61003 closed
>> >> > > > > >  connection.
>> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup
>> and
>> >> > bsp
>> >> > > > > > function.
>> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
>> >> > > > 10.131.201.14:61003
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> >> > > > > >         at
>> >> > > > > >
>> >> > >
>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> >> > > > > >         at
>> >> > > > > >
>> >> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
>> >> > > > > > ava:83)
>> >> > > > > >         at
>> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > >
>> >> >
>> >>
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> >> > > > > >         at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> >> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > >
>> >> >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> >> > > > > > Caused by: java.net.ConnectException: Connection refused
>> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> >> Method)
>> >> > > > > >         at
>> >> > > > > >
>> >> > >
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected
>> >> exception
>> >> > > from
>> >> > > > > > downstream.
>> >> > > > > > java.net.ConnectException: Connection refused
>> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> >> Method)
>> >> > > > > >  at
>> >> > > > >
>> >> >
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
>> >> > > 0x139b47286bf000c
>> >> > > > > > closed
>> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut
>> >> down
>> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping
>> service.
>> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
>> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
>> >> > > > 10.131.201.14:61003
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> >> > > > > >         at
>> >> > > > > >
>> >> > >
>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> >> > > > > >         at
>> >> > > > > >
>> >> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>> >> > > > > >         at
>> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > >
>> >> >
>> >>
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> >> > > > > >         at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> >> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > >
>> >> >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> >> > > > > > Caused by: java.net.ConnectException: Connection refused
>> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> >> Method)
>> >> > > > > >         at
>> >> > > > > >
>> >> > >
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
>> >> > > > 10.131.201.14:61003
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> >> > > > > >         at
>> >> > > > > >
>> >> > >
>> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> >> > > > > >         at
>> >> > > > > >
>> >> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>> >> > > > > >         at
>> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > >
>> >> >
>> >>
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> >> > > > > >         at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> >> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > >
>> >> >
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> >> > > > > > Caused by: java.net.ConnectException: Connection refused
>> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> >> Method)
>> >> > > > > >         at
>> >> > > > > >
>> >> > >
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> >> > > > > >         at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> >> > > > > > .....
>> >> > > > > >
>> >> > > > > > Can anybody help? I am really desparate...
>> >> > > > > >
>> >> > > > > > Thanks in advance,
>> >> > > > > >
>> >> > > > > > Sandy
>> >> > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>> >
>>
>
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
It actually failed on another node brick5 too.
I do have the write permission of /tmp dir.
But the most weird thing is:
 the snappy..so file is generated every time I run the example, but its
owner is another user who has once run hama on the same machine, even when
I manually rm and re-generate the file! And I don't have the permission to
rm it!

2012/9/11 Thomas Jungblut <th...@gmail.com>

> Oh, or just remove the snappy compression from the configuration ;)
>
> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>
> > Can you give him permission to delete in /tmp/?
> > Why is it not failing on the other hosts?
> > Otherwise you have to manually add the snappy library to the lib path of
> > the task.
> >
> >
> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >
> >> the IOException stack:
> >>
> >> java.io.IOException: failed to remove existing native library file:
> >> /tmp/snappy-1.0.4.1-libsna
> >> ppyjava.so
> >>         at
> >> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
> >>         at
> >> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
> >>         at
> >> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
> >>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> >>         at
> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> >>         at
> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> >> or.java:43)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> >> gerImpl.java:135)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> >> java:79)
> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>         at
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >>         at
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> java.lang.reflect.InvocationTargetException
> >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>         at
> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>         at
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> >> 5)
> >>         at java.lang.reflect.Method.invoke(Method.java:597)
> >>         at
> >> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> >>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> >>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> >>         at
> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
> >>         at
> >> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>         at
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >>         at
> >> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
> >> java.library.path
> >>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> >>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
> >>         at java.lang.System.loadLibrary(System.java:1028)
> >>         at
> >>
> >>
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> >>         ... 17 more
> >> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and bsp
> >> function.
> >> java.lang.NullPointerException
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
> >>         at
> >>
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
> >>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >>
> >> the user that I currently run hama doesn't have  permissions to remove
> >> /tmp/.  files.
> >> but I cannot run hama using sudo.
> >> doesn't what to do...
> >>
> >> 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >>
> >> > If you don't have additional stack information it is difficult to
> guess.
> >> > According to
> >> > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> >> > it should throw an IOException that contains the underlying error. If
> >> the
> >> > shared object can't be deleted, I would try to start by checking
> rights
> >> or
> >> > what on the other machines is not like on the machine that failed.
> >> >
> >> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >> >
> >> > > Any possible reasons for that?
> >> > >
> >> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > that is actually a good question.
> >> > > > According to the exception and the code [1] it was unable to
> delete
> >> the
> >> > > > file.
> >> > > > Actually this could have various reasons, what operating system
> are
> >> you
> >> > > > running?
> >> > > >
> >> > > > [1]
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> >> > > >
> >> > > >
> >> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >> > > >
> >> > > > > What actually happened in brick4 (the node that refused
> >> > r910(master)'s
> >> > > > > connection) is that it
> >> > > > > " failed to remove existing native library file: /tmp/
> >> > > > > snappy-1.0.4.1-libsnappyjava.so
> >> > > > >         at
> >> > > > >
> >> > >
> >>
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> >> > > > > then
> >> > > > > java.lang.NullPointerException
> >> > > > >         at
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >> > > > >
> >> > > > > But I actually run in the Hama directory and
> >> snappy-java-1.0.4.1.jar
> >> > is
> >> > > > > included in hama/lib dir.
> >> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in
> CLASSPATH.
> >> > > > > What happened?
> >> > > > >
> >> > > > >
> >> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >> > > > >
> >> > > > > > Hi, all,
> >> > > > > >
> >> > > > > > I newly set up a 3-node hama cluster following the
> >> > > > HamaInstallationGuide.
> >> > > > > > But I got some confusing errors when running the pi examples,
> >> the
> >> > > > tasklog
> >> > > > > > is as follows:
> >> > > > > >
> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
> >> > > > connection,
> >> > > > > > connectString=brick4:
> >> > > > > > 21810,r910:21810 sessionTimeout=1200000
> >> > > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> >> > > > > > l@e61a35
> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket
> >> > > connection
> >> > > > to
> >> > > > > > server brick4/10.131.
> >> > > > > > 201.14:21810
> >> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start
> >> > connecting
> >> > > > to
> >> > > > > > Zookeeper! At r910.ppi
> >> > > > > > /10.131.201.90:61001
> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
> >> > > > > established
> >> > > > > > to brick4/10.131.201
> >> > > > > > .14:21810, initiating session
> >> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
> >> establishment
> >> > > > > > complete on server brick4/10
> >> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated
> >> > timeout
> >> > > =
> >> > > > > > 1200000
> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to
> >> > > brick4.r715/
> >> > > > > > 10.131.201.14:61003
> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
> >> OPEN
> >> > > > > > java.net.ConnectException: Connection refused
> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> Method)
> >> > > > > >         at
> >> > > > > >
> >> > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> >> > > > > > tSocketPipelineSink.java:384)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> >> > > > > > eys(NioClientSocketPipelineSink.java:354)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> >> > > > > > ketPipelineSink.java:276)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
> >> > CLOSED
> >> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer
> >> > brick4.r715/
> >> > > > > > 10.131.201.14:61003 closed
> >> > > > > >  connection.
> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup
> and
> >> > bsp
> >> > > > > > function.
> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> >> > > > 10.131.201.14:61003
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >> > > > > >         at
> >> > > > > >
> >> > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >> > > > > >         at
> >> > > > > >
> >> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> >> > > > > > ava:83)
> >> > > > > >         at
> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> > > > > >         at
> >> > > > > >
> >> > > >
> >> >
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> > > > > >         at
> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> > > > > >         at
> >> > > > > >
> >> > > >
> >> >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> Method)
> >> > > > > >         at
> >> > > > > >
> >> > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected
> >> exception
> >> > > from
> >> > > > > > downstream.
> >> > > > > > java.net.ConnectException: Connection refused
> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> Method)
> >> > > > > >  at
> >> > > > >
> >> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> >> > > 0x139b47286bf000c
> >> > > > > > closed
> >> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut
> >> down
> >> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping
> service.
> >> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> >> > > > 10.131.201.14:61003
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >> > > > > >         at
> >> > > > > >
> >> > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >> > > > > >         at
> >> > > > > >
> >> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >> > > > > >         at
> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> > > > > >         at
> >> > > > > >
> >> > > >
> >> >
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> > > > > >         at
> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> > > > > >         at
> >> > > > > >
> >> > > >
> >> >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> Method)
> >> > > > > >         at
> >> > > > > >
> >> > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> > > > > >         at java.lang.Thread.run(Thread.java:662)
> >> > > > > > java.io.IOException: Error connecting to brick4.r715/
> >> > > > 10.131.201.14:61003
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >> > > > > >         at
> >> > > > > >
> >> > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >> > > > > >         at
> >> > > > > >
> >> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >> > > > > >         at
> >> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >> > > > > >         at
> >> > > > > >
> >> > > >
> >> >
> >>
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >> > > > > >         at
> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >> > > > > >         at
> >> > > > > >
> >> > > >
> >> >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> >> > > > > > Caused by: java.net.ConnectException: Connection refused
> >> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> >> Method)
> >> > > > > >         at
> >> > > > > >
> >> > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >> > > > > >         at
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> > > > > > .....
> >> > > > > >
> >> > > > > > Can anybody help? I am really desparate...
> >> > > > > >
> >> > > > > > Thanks in advance,
> >> > > > > >
> >> > > > > > Sandy
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Thomas Jungblut <th...@gmail.com>.
Oh, or just remove the snappy compression from the configuration ;)

2012/9/11 Thomas Jungblut <th...@gmail.com>

> Can you give him permission to delete in /tmp/?
> Why is it not failing on the other hosts?
> Otherwise you have to manually add the snappy library to the lib path of
> the task.
>
>
> 2012/9/11 Sandy Ding <sa...@gmail.com>
>
>> the IOException stack:
>>
>> java.io.IOException: failed to remove existing native library file:
>> /tmp/snappy-1.0.4.1-libsna
>> ppyjava.so
>>         at
>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
>>         at
>> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
>>         at
>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
>>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>>         at
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>>         at
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>>         at
>>
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
>> or.java:43)
>>         at
>>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
>> gerImpl.java:135)
>>         at
>>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
>> java:79)
>>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>         at
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>         at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> java.lang.reflect.InvocationTargetException
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
>> 5)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at
>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
>>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>>         at
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>>         at
>> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>>         at
>>
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
>>         at
>>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>>         at
>>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>         at
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>         at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
>> java.library.path
>>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
>>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
>>         at java.lang.System.loadLibrary(System.java:1028)
>>         at
>>
>> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
>>         ... 17 more
>> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and bsp
>> function.
>> java.lang.NullPointerException
>>         at
>>
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>>         at
>>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>>         at
>>
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>>
>> the user that I currently run hama doesn't have  permissions to remove
>> /tmp/.  files.
>> but I cannot run hama using sudo.
>> doesn't what to do...
>>
>> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>>
>> > If you don't have additional stack information it is difficult to guess.
>> > According to
>> > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
>> > it should throw an IOException that contains the underlying error. If
>> the
>> > shared object can't be deleted, I would try to start by checking rights
>> or
>> > what on the other machines is not like on the machine that failed.
>> >
>> > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> >
>> > > Any possible reasons for that?
>> > >
>> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
>> > >
>> > > > Hi,
>> > > >
>> > > > that is actually a good question.
>> > > > According to the exception and the code [1] it was unable to delete
>> the
>> > > > file.
>> > > > Actually this could have various reasons, what operating system are
>> you
>> > > > running?
>> > > >
>> > > > [1]
>> > > >
>> > > >
>> > >
>> >
>> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
>> > > >
>> > > >
>> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > > >
>> > > > > What actually happened in brick4 (the node that refused
>> > r910(master)'s
>> > > > > connection) is that it
>> > > > > " failed to remove existing native library file: /tmp/
>> > > > > snappy-1.0.4.1-libsnappyjava.so
>> > > > >         at
>> > > > >
>> > >
>> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
>> > > > > then
>> > > > > java.lang.NullPointerException
>> > > > >         at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>> > > > >
>> > > > > But I actually run in the Hama directory and
>> snappy-java-1.0.4.1.jar
>> > is
>> > > > > included in hama/lib dir.
>> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
>> > > > > What happened?
>> > > > >
>> > > > >
>> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
>> > > > >
>> > > > > > Hi, all,
>> > > > > >
>> > > > > > I newly set up a 3-node hama cluster following the
>> > > > HamaInstallationGuide.
>> > > > > > But I got some confusing errors when running the pi examples,
>> the
>> > > > tasklog
>> > > > > > is as follows:
>> > > > > >
>> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
>> > > > connection,
>> > > > > > connectString=brick4:
>> > > > > > 21810,r910:21810 sessionTimeout=1200000
>> > > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
>> > > > > > l@e61a35
>> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket
>> > > connection
>> > > > to
>> > > > > > server brick4/10.131.
>> > > > > > 201.14:21810
>> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start
>> > connecting
>> > > > to
>> > > > > > Zookeeper! At r910.ppi
>> > > > > > /10.131.201.90:61001
>> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
>> > > > > established
>> > > > > > to brick4/10.131.201
>> > > > > > .14:21810, initiating session
>> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
>> establishment
>> > > > > > complete on server brick4/10
>> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated
>> > timeout
>> > > =
>> > > > > > 1200000
>> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to
>> > > brick4.r715/
>> > > > > > 10.131.201.14:61003
>> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
>> OPEN
>> > > > > > java.net.ConnectException: Connection refused
>> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> Method)
>> > > > > >         at
>> > > > > >
>> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
>> > > > > > tSocketPipelineSink.java:384)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
>> > > > > > eys(NioClientSocketPipelineSink.java:354)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
>> > > > > > ketPipelineSink.java:276)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
>> > CLOSED
>> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer
>> > brick4.r715/
>> > > > > > 10.131.201.14:61003 closed
>> > > > > >  connection.
>> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and
>> > bsp
>> > > > > > function.
>> > > > > > java.io.IOException: Error connecting to brick4.r715/
>> > > > 10.131.201.14:61003
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> > > > > >         at
>> > > > > >
>> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> > > > > >         at
>> > > > > >
>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
>> > > > > > ava:83)
>> > > > > >         at
>> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > > >         at
>> > > > > >
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > > >         at
>> > > > > >
>> > > >
>> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > > > Caused by: java.net.ConnectException: Connection refused
>> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> Method)
>> > > > > >         at
>> > > > > >
>> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected
>> exception
>> > > from
>> > > > > > downstream.
>> > > > > > java.net.ConnectException: Connection refused
>> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> Method)
>> > > > > >  at
>> > > > >
>> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
>> > > 0x139b47286bf000c
>> > > > > > closed
>> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut
>> down
>> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
>> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
>> > > > > > java.io.IOException: Error connecting to brick4.r715/
>> > > > 10.131.201.14:61003
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> > > > > >         at
>> > > > > >
>> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> > > > > >         at
>> > > > > >
>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>> > > > > >         at
>> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > > >         at
>> > > > > >
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > > >         at
>> > > > > >
>> > > >
>> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > > > Caused by: java.net.ConnectException: Connection refused
>> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> Method)
>> > > > > >         at
>> > > > > >
>> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > > > > >         at java.lang.Thread.run(Thread.java:662)
>> > > > > > java.io.IOException: Error connecting to brick4.r715/
>> > > > 10.131.201.14:61003
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>> > > > > >         at
>> > > > > >
>> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>> > > > > >         at
>> > > > > >
>> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>> > > > > >         at
>> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>> > > > > >         at
>> > > > > >
>> > > >
>> >
>> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>> > > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>> > > > > >         at
>> > > > > >
>> > > >
>> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
>> > > > > > Caused by: java.net.ConnectException: Connection refused
>> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
>> Method)
>> > > > > >         at
>> > > > > >
>> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>> > > > > >         at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > > > > > .....
>> > > > > >
>> > > > > > Can anybody help? I am really desparate...
>> > > > > >
>> > > > > > Thanks in advance,
>> > > > > >
>> > > > > > Sandy
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: my new hama cluster cannot run pi example

Posted by Thomas Jungblut <th...@gmail.com>.
Can you give him permission to delete in /tmp/?
Why is it not failing on the other hosts?
Otherwise you have to manually add the snappy library to the lib path of
the task.

2012/9/11 Sandy Ding <sa...@gmail.com>

> the IOException stack:
>
> java.io.IOException: failed to remove existing native library file:
> /tmp/snappy-1.0.4.1-libsna
> ppyjava.so
>         at
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
>         at
> org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
>         at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>         at
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>         at
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>         at
>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
> or.java:43)
>         at
>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
> gerImpl.java:135)
>         at
>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
> java:79)
>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>         at
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>         at
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
> 5)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
>         at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
>         at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
>         at
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
>         at
> org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
>         at
>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
>         at
>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>         at
>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>         at
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>         at
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
> java.library.path
>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
>         at java.lang.System.loadLibrary(System.java:1028)
>         at
>
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
>         ... 17 more
> 12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and bsp
> function.
> java.lang.NullPointerException
>         at
>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>         at
>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
>         at
>
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>
> the user that I currently run hama doesn't have  permissions to remove
> /tmp/.  files.
> but I cannot run hama using sudo.
> doesn't what to do...
>
> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>
> > If you don't have additional stack information it is difficult to guess.
> > According to
> > http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> > it should throw an IOException that contains the underlying error. If the
> > shared object can't be deleted, I would try to start by checking rights
> or
> > what on the other machines is not like on the machine that failed.
> >
> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >
> > > Any possible reasons for that?
> > >
> > > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> > >
> > > > Hi,
> > > >
> > > > that is actually a good question.
> > > > According to the exception and the code [1] it was unable to delete
> the
> > > > file.
> > > > Actually this could have various reasons, what operating system are
> you
> > > > running?
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> > > >
> > > >
> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > >
> > > > > What actually happened in brick4 (the node that refused
> > r910(master)'s
> > > > > connection) is that it
> > > > > " failed to remove existing native library file: /tmp/
> > > > > snappy-1.0.4.1-libsnappyjava.so
> > > > >         at
> > > > >
> > >
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > > > > then
> > > > > java.lang.NullPointerException
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > > > >
> > > > > But I actually run in the Hama directory and
> snappy-java-1.0.4.1.jar
> > is
> > > > > included in hama/lib dir.
> > > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
> > > > > What happened?
> > > > >
> > > > >
> > > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > > >
> > > > > > Hi, all,
> > > > > >
> > > > > > I newly set up a 3-node hama cluster following the
> > > > HamaInstallationGuide.
> > > > > > But I got some confusing errors when running the pi examples, the
> > > > tasklog
> > > > > > is as follows:
> > > > > >
> > > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
> > > > connection,
> > > > > > connectString=brick4:
> > > > > > 21810,r910:21810 sessionTimeout=1200000
> > > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > > > > > l@e61a35
> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket
> > > connection
> > > > to
> > > > > > server brick4/10.131.
> > > > > > 201.14:21810
> > > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start
> > connecting
> > > > to
> > > > > > Zookeeper! At r910.ppi
> > > > > > /10.131.201.90:61001
> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
> > > > > established
> > > > > > to brick4/10.131.201
> > > > > > .14:21810, initiating session
> > > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session
> establishment
> > > > > > complete on server brick4/10
> > > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated
> > timeout
> > > =
> > > > > > 1200000
> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to
> > > brick4.r715/
> > > > > > 10.131.201.14:61003
> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
> OPEN
> > > > > > java.net.ConnectException: Connection refused
> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> Method)
> > > > > >         at
> > > > > >
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > > > > > tSocketPipelineSink.java:384)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > > > > > eys(NioClientSocketPipelineSink.java:354)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > > > > > ketPipelineSink.java:276)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
> > CLOSED
> > > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer
> > brick4.r715/
> > > > > > 10.131.201.14:61003 closed
> > > > > >  connection.
> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and
> > bsp
> > > > > > function.
> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > > 10.131.201.14:61003
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > > >         at
> > > > > >
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > > >         at
> > > > > >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > > > > > ava:83)
> > > > > >         at
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > > >         at
> > > > > >
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > > >         at
> > > > > >
> > > >
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > > > Caused by: java.net.ConnectException: Connection refused
> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> Method)
> > > > > >         at
> > > > > >
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception
> > > from
> > > > > > downstream.
> > > > > > java.net.ConnectException: Connection refused
> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> Method)
> > > > > >  at
> > > > >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> > > 0x139b47286bf000c
> > > > > > closed
> > > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut
> down
> > > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
> > > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > > 10.131.201.14:61003
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > > >         at
> > > > > >
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > > >         at
> > > > > >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > > > >         at
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > > >         at
> > > > > >
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > > >         at
> > > > > >
> > > >
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > > > Caused by: java.net.ConnectException: Connection refused
> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> Method)
> > > > > >         at
> > > > > >
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > > java.io.IOException: Error connecting to brick4.r715/
> > > > 10.131.201.14:61003
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > > >         at
> > > > > >
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > > >         at
> > > > > >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > > > >         at
> > org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > > >         at
> > > > > >
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > > >         at
> > > > > >
> > > >
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > > > Caused by: java.net.ConnectException: Connection refused
> > > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native
> Method)
> > > > > >         at
> > > > > >
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > > >         at
> > > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > > > .....
> > > > > >
> > > > > > Can anybody help? I am really desparate...
> > > > > >
> > > > > > Thanks in advance,
> > > > > >
> > > > > > Sandy
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
the IOException stack:

java.io.IOException: failed to remove existing native library file:
/tmp/snappy-1.0.4.1-libsna
ppyjava.so
        at
org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)
        at
org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:446)
        at
org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:308)
        at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
        at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
        at
org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
        at
org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
        at
org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompress
or.java:43)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageMana
gerImpl.java:135)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.
java:79)
        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
        at
org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
        at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
        at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
        at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
5)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
        at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
        at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
        at
org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
        at
org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
        at
org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:43)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
        at
org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
        at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
        at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
        at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in
java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at
org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
        ... 17 more
12/09/11 21:27:14 ERROR bsp.BSPTask: Error running bsp setup and bsp
function.
java.lang.NullPointerException
        at
org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.serializeMessage(AvroMessageManagerImpl.java:135)
        at
org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:79)
        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)

the user that I currently run hama doesn't have  permissions to remove
/tmp/.  files.
but I cannot run hama using sudo.
doesn't what to do...

2012/9/11 Thomas Jungblut <th...@gmail.com>

> If you don't have additional stack information it is difficult to guess.
> According to
> http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
> it should throw an IOException that contains the underlying error. If the
> shared object can't be deleted, I would try to start by checking rights or
> what on the other machines is not like on the machine that failed.
>
> 2012/9/11 Sandy Ding <sa...@gmail.com>
>
> > Any possible reasons for that?
> >
> > 2012/9/11 Thomas Jungblut <th...@gmail.com>
> >
> > > Hi,
> > >
> > > that is actually a good question.
> > > According to the exception and the code [1] it was unable to delete the
> > > file.
> > > Actually this could have various reasons, what operating system are you
> > > running?
> > >
> > > [1]
> > >
> > >
> >
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> > >
> > >
> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >
> > > > What actually happened in brick4 (the node that refused
> r910(master)'s
> > > > connection) is that it
> > > > " failed to remove existing native library file: /tmp/
> > > > snappy-1.0.4.1-libsnappyjava.so
> > > >         at
> > > >
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > > > then
> > > > java.lang.NullPointerException
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > > >
> > > > But I actually run in the Hama directory and snappy-java-1.0.4.1.jar
> is
> > > > included in hama/lib dir.
> > > > I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
> > > > What happened?
> > > >
> > > >
> > > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > > >
> > > > > Hi, all,
> > > > >
> > > > > I newly set up a 3-node hama cluster following the
> > > HamaInstallationGuide.
> > > > > But I got some confusing errors when running the pi examples, the
> > > tasklog
> > > > > is as follows:
> > > > >
> > > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
> > > connection,
> > > > > connectString=brick4:
> > > > > 21810,r910:21810 sessionTimeout=1200000
> > > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > > > > l@e61a35
> > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket
> > connection
> > > to
> > > > > server brick4/10.131.
> > > > > 201.14:21810
> > > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start
> connecting
> > > to
> > > > > Zookeeper! At r910.ppi
> > > > > /10.131.201.90:61001
> > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
> > > > established
> > > > > to brick4/10.131.201
> > > > > .14:21810, initiating session
> > > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session establishment
> > > > > complete on server brick4/10
> > > > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated
> timeout
> > =
> > > > > 1200000
> > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to
> > brick4.r715/
> > > > > 10.131.201.14:61003
> > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] OPEN
> > > > > java.net.ConnectException: Connection refused
> > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > > >         at
> > > > >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > > > > tSocketPipelineSink.java:384)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > > > > eys(NioClientSocketPipelineSink.java:354)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > > > > ketPipelineSink.java:276)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a]
> CLOSED
> > > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer
> brick4.r715/
> > > > > 10.131.201.14:61003 closed
> > > > >  connection.
> > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and
> bsp
> > > > > function.
> > > > > java.io.IOException: Error connecting to brick4.r715/
> > > 10.131.201.14:61003
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > >         at
> > > > >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > >         at
> > > > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > > > > ava:83)
> > > > >         at
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >         at
> > > > >
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >         at
> > > > >
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > > Caused by: java.net.ConnectException: Connection refused
> > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > > >         at
> > > > >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception
> > from
> > > > > downstream.
> > > > > java.net.ConnectException: Connection refused
> > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > > >  at
> > > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> > 0x139b47286bf000c
> > > > > closed
> > > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut down
> > > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
> > > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> > > > > java.io.IOException: Error connecting to brick4.r715/
> > > 10.131.201.14:61003
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > >         at
> > > > >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > >         at
> > > > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > > >         at
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >         at
> > > > >
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >         at
> > > > >
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > > Caused by: java.net.ConnectException: Connection refused
> > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > > >         at
> > > > >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > > >         at java.lang.Thread.run(Thread.java:662)
> > > > > java.io.IOException: Error connecting to brick4.r715/
> > > 10.131.201.14:61003
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > > >         at
> > > > >
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > > >         at
> > > > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > > >         at
> org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > > >         at
> > > > >
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > > >         at
> > > > >
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > > Caused by: java.net.ConnectException: Connection refused
> > > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > > >         at
> > > > >
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > > >         at
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > > .....
> > > > >
> > > > > Can anybody help? I am really desparate...
> > > > >
> > > > > Thanks in advance,
> > > > >
> > > > > Sandy
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Thomas Jungblut <th...@gmail.com>.
If you don't have additional stack information it is difficult to guess.
According to
http://docs.oracle.com/javase/7/docs/api/java/io/File.html#delete()
it should throw an IOException that contains the underlying error. If the
shared object can't be deleted, I would try to start by checking rights or
what on the other machines is not like on the machine that failed.

2012/9/11 Sandy Ding <sa...@gmail.com>

> Any possible reasons for that?
>
> 2012/9/11 Thomas Jungblut <th...@gmail.com>
>
> > Hi,
> >
> > that is actually a good question.
> > According to the exception and the code [1] it was unable to delete the
> > file.
> > Actually this could have various reasons, what operating system are you
> > running?
> >
> > [1]
> >
> >
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
> >
> >
> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >
> > > What actually happened in brick4 (the node that refused r910(master)'s
> > > connection) is that it
> > > " failed to remove existing native library file: /tmp/
> > > snappy-1.0.4.1-libsnappyjava.so
> > >         at
> > >
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > > then
> > > java.lang.NullPointerException
> > >         at
> > >
> > >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> > >
> > > But I actually run in the Hama directory and snappy-java-1.0.4.1.jar is
> > > included in hama/lib dir.
> > > I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
> > > What happened?
> > >
> > >
> > > 2012/9/11 Sandy Ding <sa...@gmail.com>
> > >
> > > > Hi, all,
> > > >
> > > > I newly set up a 3-node hama cluster following the
> > HamaInstallationGuide.
> > > > But I got some confusing errors when running the pi examples, the
> > tasklog
> > > > is as follows:
> > > >
> > > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
> > connection,
> > > > connectString=brick4:
> > > > 21810,r910:21810 sessionTimeout=1200000
> > > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > > > l@e61a35
> > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket
> connection
> > to
> > > > server brick4/10.131.
> > > > 201.14:21810
> > > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start connecting
> > to
> > > > Zookeeper! At r910.ppi
> > > > /10.131.201.90:61001
> > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
> > > established
> > > > to brick4/10.131.201
> > > > .14:21810, initiating session
> > > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session establishment
> > > > complete on server brick4/10
> > > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated timeout
> =
> > > > 1200000
> > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to
> brick4.r715/
> > > > 10.131.201.14:61003
> > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] OPEN
> > > > java.net.ConnectException: Connection refused
> > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > >         at
> > > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > > > tSocketPipelineSink.java:384)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > > > eys(NioClientSocketPipelineSink.java:354)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > > > ketPipelineSink.java:276)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >         at java.lang.Thread.run(Thread.java:662)
> > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] CLOSED
> > > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer brick4.r715/
> > > > 10.131.201.14:61003 closed
> > > >  connection.
> > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and bsp
> > > > function.
> > > > java.io.IOException: Error connecting to brick4.r715/
> > 10.131.201.14:61003
> > > >         at
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > >         at
> > > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > >         at
> > > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > >         at
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > > > ava:83)
> > > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >         at
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >         at
> > > >
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > Caused by: java.net.ConnectException: Connection refused
> > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > >         at
> > > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >         at java.lang.Thread.run(Thread.java:662)
> > > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception
> from
> > > > downstream.
> > > > java.net.ConnectException: Connection refused
> > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > >  at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >         at java.lang.Thread.run(Thread.java:662)
> > > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session:
> 0x139b47286bf000c
> > > > closed
> > > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut down
> > > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
> > > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> > > > java.io.IOException: Error connecting to brick4.r715/
> > 10.131.201.14:61003
> > > >         at
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > >         at
> > > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > >         at
> > > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > >         at
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >         at
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >         at
> > > >
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > Caused by: java.net.ConnectException: Connection refused
> > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > >         at
> > > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > > >         at java.lang.Thread.run(Thread.java:662)
> > > > java.io.IOException: Error connecting to brick4.r715/
> > 10.131.201.14:61003
> > > >         at
> > > >
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > > >         at
> > > >
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > > >         at
> > > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > > >         at
> > > >
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > > >         at
> > > >
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > > >         at
> > > >
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > > Caused by: java.net.ConnectException: Connection refused
> > > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > >         at
> > > >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > > >         at
> > > >
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > > >         at
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > > .....
> > > >
> > > > Can anybody help? I am really desparate...
> > > >
> > > > Thanks in advance,
> > > >
> > > > Sandy
> > > >
> > > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
Any possible reasons for that?

2012/9/11 Thomas Jungblut <th...@gmail.com>

> Hi,
>
> that is actually a good question.
> According to the exception and the code [1] it was unable to delete the
> file.
> Actually this could have various reasons, what operating system are you
> running?
>
> [1]
>
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
>
>
> 2012/9/11 Sandy Ding <sa...@gmail.com>
>
> > What actually happened in brick4 (the node that refused r910(master)'s
> > connection) is that it
> > " failed to remove existing native library file: /tmp/
> > snappy-1.0.4.1-libsnappyjava.so
> >         at
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > then
> > java.lang.NullPointerException
> >         at
> >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >
> > But I actually run in the Hama directory and snappy-java-1.0.4.1.jar is
> > included in hama/lib dir.
> > I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
> > What happened?
> >
> >
> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >
> > > Hi, all,
> > >
> > > I newly set up a 3-node hama cluster following the
> HamaInstallationGuide.
> > > But I got some confusing errors when running the pi examples, the
> tasklog
> > > is as follows:
> > >
> > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> > > connectString=brick4:
> > > 21810,r910:21810 sessionTimeout=1200000
> > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > > l@e61a35
> > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket connection
> to
> > > server brick4/10.131.
> > > 201.14:21810
> > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start connecting
> to
> > > Zookeeper! At r910.ppi
> > > /10.131.201.90:61001
> > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
> > established
> > > to brick4/10.131.201
> > > .14:21810, initiating session
> > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session establishment
> > > complete on server brick4/10
> > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated timeout =
> > > 1200000
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to brick4.r715/
> > > 10.131.201.14:61003
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] OPEN
> > > java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > > tSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > > eys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > > ketPipelineSink.java:276)
> > >         at
> > >
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > >         at
> > >
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] CLOSED
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer brick4.r715/
> > > 10.131.201.14:61003 closed
> > >  connection.
> > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and bsp
> > > function.
> > > java.io.IOException: Error connecting to brick4.r715/
> 10.131.201.14:61003
> > >         at
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >         at
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > > ava:83)
> > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >         at
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >         at
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > Caused by: java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception from
> > > downstream.
> > > java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >  at
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session: 0x139b47286bf000c
> > > closed
> > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut down
> > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
> > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> > > java.io.IOException: Error connecting to brick4.r715/
> 10.131.201.14:61003
> > >         at
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >         at
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >         at
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >         at
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > Caused by: java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > java.io.IOException: Error connecting to brick4.r715/
> 10.131.201.14:61003
> > >         at
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >         at
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >         at
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >         at
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > Caused by: java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > .....
> > >
> > > Can anybody help? I am really desparate...
> > >
> > > Thanks in advance,
> > >
> > > Sandy
> > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
Debian 6.0

2012/9/11 Thomas Jungblut <th...@gmail.com>

> Hi,
>
> that is actually a good question.
> According to the exception and the code [1] it was unable to delete the
> file.
> Actually this could have various reasons, what operating system are you
> running?
>
> [1]
>
> https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374
>
>
> 2012/9/11 Sandy Ding <sa...@gmail.com>
>
> > What actually happened in brick4 (the node that refused r910(master)'s
> > connection) is that it
> > " failed to remove existing native library file: /tmp/
> > snappy-1.0.4.1-libsnappyjava.so
> >         at
> > org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> > then
> > java.lang.NullPointerException
> >         at
> >
> >
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
> >
> > But I actually run in the Hama directory and snappy-java-1.0.4.1.jar is
> > included in hama/lib dir.
> > I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
> > What happened?
> >
> >
> > 2012/9/11 Sandy Ding <sa...@gmail.com>
> >
> > > Hi, all,
> > >
> > > I newly set up a 3-node hama cluster following the
> HamaInstallationGuide.
> > > But I got some confusing errors when running the pi examples, the
> tasklog
> > > is as follows:
> > >
> > > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> > > connectString=brick4:
> > > 21810,r910:21810 sessionTimeout=1200000
> > > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > > l@e61a35
> > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket connection
> to
> > > server brick4/10.131.
> > > 201.14:21810
> > > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start connecting
> to
> > > Zookeeper! At r910.ppi
> > > /10.131.201.90:61001
> > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
> > established
> > > to brick4/10.131.201
> > > .14:21810, initiating session
> > > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session establishment
> > > complete on server brick4/10
> > > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated timeout =
> > > 1200000
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to brick4.r715/
> > > 10.131.201.14:61003
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] OPEN
> > > java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > > tSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > > eys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > > ketPipelineSink.java:276)
> > >         at
> > >
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > >         at
> > >
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] CLOSED
> > > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer brick4.r715/
> > > 10.131.201.14:61003 closed
> > >  connection.
> > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and bsp
> > > function.
> > > java.io.IOException: Error connecting to brick4.r715/
> 10.131.201.14:61003
> > >         at
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >         at
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > > ava:83)
> > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >         at
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >         at
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > Caused by: java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception from
> > > downstream.
> > > java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >  at
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session: 0x139b47286bf000c
> > > closed
> > > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut down
> > > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
> > > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> > > java.io.IOException: Error connecting to brick4.r715/
> 10.131.201.14:61003
> > >         at
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >         at
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >         at
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >         at
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > Caused by: java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > java.io.IOException: Error connecting to brick4.r715/
> 10.131.201.14:61003
> > >         at
> > >
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> > >         at
> > > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> > >         at
> > >
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> > >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> > >         at
> > >
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> > >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> > >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> > >         at
> > >
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > > Caused by: java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> > >         at
> > >
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> > >         at
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > > .....
> > >
> > > Can anybody help? I am really desparate...
> > >
> > > Thanks in advance,
> > >
> > > Sandy
> > >
> > >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Thomas Jungblut <th...@gmail.com>.
Hi,

that is actually a good question.
According to the exception and the code [1] it was unable to delete the
file.
Actually this could have various reasons, what operating system are you
running?

[1]
https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L374


2012/9/11 Sandy Ding <sa...@gmail.com>

> What actually happened in brick4 (the node that refused r910(master)'s
> connection) is that it
> " failed to remove existing native library file: /tmp/
> snappy-1.0.4.1-libsnappyjava.so
>         at
> org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
> then
> java.lang.NullPointerException
>         at
>
> org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)
>
> But I actually run in the Hama directory and snappy-java-1.0.4.1.jar is
> included in hama/lib dir.
> I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
> What happened?
>
>
> 2012/9/11 Sandy Ding <sa...@gmail.com>
>
> > Hi, all,
> >
> > I newly set up a 3-node hama cluster following the HamaInstallationGuide.
> > But I got some confusing errors when running the pi examples, the tasklog
> > is as follows:
> >
> > 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client connection,
> > connectString=brick4:
> > 21810,r910:21810 sessionTimeout=1200000
> > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> > l@e61a35
> > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket connection to
> > server brick4/10.131.
> > 201.14:21810
> > 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
> > Zookeeper! At r910.ppi
> > /10.131.201.90:61001
> > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection
> established
> > to brick4/10.131.201
> > .14:21810, initiating session
> > 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session establishment
> > complete on server brick4/10
> > .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated timeout =
> > 1200000
> > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to brick4.r715/
> > 10.131.201.14:61003
> > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] OPEN
> > java.net.ConnectException: Connection refused
> >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> >         at
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> > tSocketPipelineSink.java:384)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> > eys(NioClientSocketPipelineSink.java:354)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> > ketPipelineSink.java:276)
> >         at
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> >         at
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >         at java.lang.Thread.run(Thread.java:662)
> > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] CLOSED
> > 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer brick4.r715/
> > 10.131.201.14:61003 closed
> >  connection.
> > 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and bsp
> > function.
> > java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
> >         at
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >         at
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >         at
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >         at
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> > ava:83)
> >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >         at
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >         at
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > Caused by: java.net.ConnectException: Connection refused
> >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> >         at
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >         at java.lang.Thread.run(Thread.java:662)
> > 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception from
> > downstream.
> > java.net.ConnectException: Connection refused
> >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> >  at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >         at java.lang.Thread.run(Thread.java:662)
> > 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session: 0x139b47286bf000c
> > closed
> > 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut down
> > 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
> > 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> > java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
> >         at
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >         at
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >         at
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >         at
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >         at
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >         at
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > Caused by: java.net.ConnectException: Connection refused
> >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> >         at
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >         at java.lang.Thread.run(Thread.java:662)
> > java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
> >         at
> >
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
> >         at
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
> >         at
> > org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
> >         at
> >
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
> >         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
> >         at
> > org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
> >         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
> >         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
> >         at
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> > Caused by: java.net.ConnectException: Connection refused
> >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> >         at
> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
> >         at
> >
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > .....
> >
> > Can anybody help? I am really desparate...
> >
> > Thanks in advance,
> >
> > Sandy
> >
> >
>

Re: my new hama cluster cannot run pi example

Posted by Sandy Ding <sa...@gmail.com>.
What actually happened in brick4 (the node that refused r910(master)'s
connection) is that it
" failed to remove existing native library file: /tmp/
snappy-1.0.4.1-libsnappyjava.so
        at
org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:376)"
then
java.lang.NullPointerException
        at
org.apache.hama.bsp.message.compress.SnappyCompressor.compressBundle(SnappyCompressor.java:56)

But I actually run in the Hama directory and snappy-java-1.0.4.1.jar is
included in hama/lib dir.
I even include the hama/lib/snappy-java-1.0.4.1.jar in CLASSPATH.
What happened?


2012/9/11 Sandy Ding <sa...@gmail.com>

> Hi, all,
>
> I newly set up a 3-node hama cluster following the HamaInstallationGuide.
> But I got some confusing errors when running the pi examples, the tasklog
> is as follows:
>
> 12/09/11 19:23:39 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=brick4:
> 21810,r910:21810 sessionTimeout=1200000
> watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImp
> l@e61a35
> 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Opening socket connection to
> server brick4/10.131.
> 201.14:21810
> 12/09/11 19:23:39 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
> Zookeeper! At r910.ppi
> /10.131.201.90:61001
> 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Socket connection established
> to brick4/10.131.201
> .14:21810, initiating session
> 12/09/11 19:23:39 INFO zookeeper.ClientCnxn: Session establishment
> complete on server brick4/10
> .131.201.14:21810, sessionid = 0x139b47286bf000c, negotiated timeout =
> 1200000
> 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Connecting to brick4.r715/
> 10.131.201.14:61003
> 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] OPEN
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClien
> tSocketPipelineSink.java:384)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedK
> eys(NioClientSocketPipelineSink.java:354)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSoc
> ketPipelineSink.java:276)
>         at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>         at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> 12/09/11 19:23:40 INFO ipc.NettyTransceiver: [id: 0x0118278a] CLOSED
> 12/09/11 19:23:40 INFO ipc.NettyTransceiver: Remote peer brick4.r715/
> 10.131.201.14:61003 closed
>  connection.
> 12/09/11 19:23:40 ERROR bsp.BSPTask: Error running bsp setup and bsp
> function.
> java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
>         at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>         at
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.j
> ava:83)
>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>         at
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>         at
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> Caused by: java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> 12/09/11 19:23:40 WARN ipc.NettyTransceiver: Unexpected exception from
> downstream.
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> 12/09/11 19:23:40 INFO zookeeper.ZooKeeper: Session: 0x139b47286bf000c
> closed
> 12/09/11 19:23:40 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/09/11 19:23:40 ERROR bsp.BSPTask: Shutting down ping service.
> 12/09/11 19:23:40 FATAL bsp.GroomServer: Error running child
> java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
>         at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>         at
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>         at
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>         at
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> Caused by: java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> java.io.IOException: Error connecting to brick4.r715/10.131.201.14:61003
>         at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:163)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:128)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:91)
>         at
> org.apache.hama.bsp.message.AvroMessageManagerImpl.transfer(AvroMessageManagerImpl.java:83)
>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:328)
>         at
> org.apache.hama.examples.PiEstimator$MyEstimator.bsp(PiEstimator.java:69)
>         at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
>         at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>         at
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1158)
> Caused by: java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> .....
>
> Can anybody help? I am really desparate...
>
> Thanks in advance,
>
> Sandy
>
>