You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by dh...@tcs.com on 2014/11/25 08:09:19 UTC
Zookeepr shuting down.
Hi
I have a cluster of 3 systems each of 32GB RAM and 1 TB HD. I have
clustered all the three and able to start and run Hadoop Successfully.
I have installed Hbase on the master node. Now am trying to start
Zookeeper in the cluster. When I start zookeeper and give command
./zkServer.sh status its telling 'Zookeeper might not be running... '.
When I start zookeeper on all Nodes, zookeper on Master node becomes
follower and when an MR is run , It will run 90% on reducer part then
throws error 'Not able to find the region server.'. When checked zookeeper
logs, Am viewing 'Connection error'.
There is proper ssh across all 3 systems. I have tried giving iptables
-flush too, but no luck!!
I have tried another option by allowing Hbase to handle its own zookeeper.
HQuarampeer starts successfully, and I am able to process 3GB file through
MR for first time. But from second run zookeeper on one of the nodes
getting killed. not able to figure out the issue.
Please help!!!
Thanks & Regards
Dhamodharan
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Re: Zookeeper shuting down.
Posted by Ted Yu <yu...@gmail.com>.
bq. Cannot open channel to 1 at election address /172.10.195.299:3888
Can you check zookeeper log on 172.10.195.299 ?
Cheers
On Thu, Nov 27, 2014 at 12:29 AM, <dh...@tcs.com> wrote:
> Hi,
>
> Please find the log from the master node, I am using hbase-0.94.12 and
> zookeeper-3.4.5
>
> 014-11-27 12:32:21,444 [myid:0] - INFO
> [Thread-1:QuorumCnxManager$Listener@486] - My election bind port:
> 0.0.0.0/0.0.0.0:3888
> 2014-11-27 12:32:21,459 [myid:0] - INFO
> [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181:QuorumPeer@670] - LOOKING
> 2014-11-27 12:32:21,461 [myid:0] - INFO
> [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@740] - New
> election. My id = 0, proposed zxid=0x40
> 2014-11-27 12:32:21,464 [myid:0] - INFO
> [WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0
> (n.leader), 0x40 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid),
> 0x1 (n.peerEPoch), LOOKING (my state)
> 2014-11-27 12:32:21,468 [myid:0] - WARN
> [WorkerSender[myid=0]:QuorumCnxManager@368] - Cannot open channel to 1 at
> election address /172.10.195.299:3888
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
>
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
>
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at
>
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:354)
> at
>
> org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:327)
> at
>
> org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393)
> at
>
> org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365)
> at java.lang.Thread.run(Thread.java:724)
>
> Thanks & Regards
> Dhamodharan Ramalingam
>
>
>
> From: Ted Yu <yu...@gmail.com>
> To: "user@hbase.apache.org" <us...@hbase.apache.org>
> Date: 11/25/2014 08:28 PM
> Subject: Re: Zookeepr shuting down.
>
>
>
> Can you pastebin zookeeper log from the master node ?
>
> What hbase / zookeeper version are you using ?
>
> Cheers
>
> On Mon, Nov 24, 2014 at 11:09 PM, <dh...@tcs.com> wrote:
>
> > Hi
> >
> > I have a cluster of 3 systems each of 32GB RAM and 1 TB HD. I have
> > clustered all the three and able to start and run Hadoop Successfully.
> >
> > I have installed Hbase on the master node. Now am trying to start
> > Zookeeper in the cluster. When I start zookeeper and give command
> > ./zkServer.sh status its telling 'Zookeeper might not be running... '.
> > When I start zookeeper on all Nodes, zookeper on Master node becomes
> > follower and when an MR is run , It will run 90% on reducer part then
> > throws error 'Not able to find the region server.'. When checked
> zookeeper
> > logs, Am viewing 'Connection error'.
> >
> > There is proper ssh across all 3 systems. I have tried giving iptables
> > -flush too, but no luck!!
> >
> > I have tried another option by allowing Hbase to handle its own
> zookeeper.
> > HQuarampeer starts successfully, and I am able to process 3GB file
> through
> > MR for first time. But from second run zookeeper on one of the nodes
> > getting killed. not able to figure out the issue.
> >
> > Please help!!!
> >
> > Thanks & Regards
> > Dhamodharan
> >
> > =====-----=====-----=====
> > Notice: The information contained in this e-mail
> > message and/or attachments to it may contain
> > confidential or privileged information. If you are
> > not the intended recipient, any dissemination, use,
> > review, distribution, printing or copying of the
> > information contained in this e-mail message
> > and/or attachments to it are strictly prohibited. If
> > you have received this communication in error,
> > please notify us by reply e-mail or telephone and
> > immediately and permanently delete the message
> > and any attachments. Thank you
> >
> >
> >
>
>
Re: Zookeeper shuting down.
Posted by dh...@tcs.com.
Hi,
Please find the log from the master node, I am using hbase-0.94.12 and
zookeeper-3.4.5
014-11-27 12:32:21,444 [myid:0] - INFO
[Thread-1:QuorumCnxManager$Listener@486] - My election bind port:
0.0.0.0/0.0.0.0:3888
2014-11-27 12:32:21,459 [myid:0] - INFO
[QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181:QuorumPeer@670] - LOOKING
2014-11-27 12:32:21,461 [myid:0] - INFO
[QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@740] - New
election. My id = 0, proposed zxid=0x40
2014-11-27 12:32:21,464 [myid:0] - INFO
[WorkerReceiver[myid=0]:FastLeaderElection@542] - Notification: 0
(n.leader), 0x40 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid),
0x1 (n.peerEPoch), LOOKING (my state)
2014-11-27 12:32:21,468 [myid:0] - WARN
[WorkerSender[myid=0]:QuorumCnxManager@368] - Cannot open channel to 1 at
election address /172.10.195.299:3888
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at
org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:354)
at
org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:327)
at
org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:393)
at
org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:365)
at java.lang.Thread.run(Thread.java:724)
Thanks & Regards
Dhamodharan Ramalingam
From: Ted Yu <yu...@gmail.com>
To: "user@hbase.apache.org" <us...@hbase.apache.org>
Date: 11/25/2014 08:28 PM
Subject: Re: Zookeepr shuting down.
Can you pastebin zookeeper log from the master node ?
What hbase / zookeeper version are you using ?
Cheers
On Mon, Nov 24, 2014 at 11:09 PM, <dh...@tcs.com> wrote:
> Hi
>
> I have a cluster of 3 systems each of 32GB RAM and 1 TB HD. I have
> clustered all the three and able to start and run Hadoop Successfully.
>
> I have installed Hbase on the master node. Now am trying to start
> Zookeeper in the cluster. When I start zookeeper and give command
> ./zkServer.sh status its telling 'Zookeeper might not be running... '.
> When I start zookeeper on all Nodes, zookeper on Master node becomes
> follower and when an MR is run , It will run 90% on reducer part then
> throws error 'Not able to find the region server.'. When checked
zookeeper
> logs, Am viewing 'Connection error'.
>
> There is proper ssh across all 3 systems. I have tried giving iptables
> -flush too, but no luck!!
>
> I have tried another option by allowing Hbase to handle its own
zookeeper.
> HQuarampeer starts successfully, and I am able to process 3GB file
through
> MR for first time. But from second run zookeeper on one of the nodes
> getting killed. not able to figure out the issue.
>
> Please help!!!
>
> Thanks & Regards
> Dhamodharan
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>
Re: Zookeepr shuting down.
Posted by Ted Yu <yu...@gmail.com>.
Can you pastebin zookeeper log from the master node ?
What hbase / zookeeper version are you using ?
Cheers
On Mon, Nov 24, 2014 at 11:09 PM, <dh...@tcs.com> wrote:
> Hi
>
> I have a cluster of 3 systems each of 32GB RAM and 1 TB HD. I have
> clustered all the three and able to start and run Hadoop Successfully.
>
> I have installed Hbase on the master node. Now am trying to start
> Zookeeper in the cluster. When I start zookeeper and give command
> ./zkServer.sh status its telling 'Zookeeper might not be running... '.
> When I start zookeeper on all Nodes, zookeper on Master node becomes
> follower and when an MR is run , It will run 90% on reducer part then
> throws error 'Not able to find the region server.'. When checked zookeeper
> logs, Am viewing 'Connection error'.
>
> There is proper ssh across all 3 systems. I have tried giving iptables
> -flush too, but no luck!!
>
> I have tried another option by allowing Hbase to handle its own zookeeper.
> HQuarampeer starts successfully, and I am able to process 3GB file through
> MR for first time. But from second run zookeeper on one of the nodes
> getting killed. not able to figure out the issue.
>
> Please help!!!
>
> Thanks & Regards
> Dhamodharan
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>