You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Ananth T. Sarathy" <an...@gmail.com> on 2009/12/03 19:52:07 UTC
Zookeeper problem
Hi,
when my regionservers try to connect, I keep getting a zookeeper timeout.
on the server zookeeper is running i get this error on the zookeeper log
2009-12-03 13:50:18,415 WARN
org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
processing sessionid:0x12555e051b10001 type:create cxid:0x8
zxid:0xfffffffffffffffe txntype:unknown n/a
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists
at
org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245)
at
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
Any ideas on how to clear this up?
Ananth T Sarathy
Re: Zookeeper problem
Posted by Jean-Daniel Cryans <jd...@apache.org>.
The best in your setup would seem to override them all. You do have a
DNS server right?
J-D
On Thu, Dec 3, 2009 at 11:40 AM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> hmm ok. i ll try that. do I only need to change the
> hbase.master.dns.nameserver and interface or should I override the zookeeper
> and regionserver as well?
> Ananth T Sarathy
>
>
> On Thu, Dec 3, 2009 at 2:23 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> Some problems I see:
>>
>> - ulimit - n 1024, the Getting Started documentation states that you
>> should set it to a much higher number or you will run into problems at
>> some point.
>>
>> - First time you start it says "java.net.NoRouteToHostException: No
>> route to host" and the address it got is "192.168.20.89:60000". Seems
>> like a network problem. This means the master reported it's address on
>> 192.168.20.89.
>>
>> - Second time it tries to connect to the zookeeper server
>> "192.169.20.89:2181" but gets a "Transport endpoint is not connected"
>> so it cannot connect to it. The timeout you get is really a connection
>> timeout while it retries to connect.
>>
>> So it seems your nodes are only able to connect when they use their
>> hostname. One solution would be to play with
>> hbase.master.dns.interface and hbase.master.dns.nameserver (see
>> hbase-default.xml) and to set the quorum address to the hostname.
>>
>> J-D
>>
>> On Thu, Dec 3, 2009 at 11:13 AM, Ananth T. Sarathy
>> <an...@gmail.com> wrote:
>> > Sure
>> >
>> > http://pastebin.com/m670c216b
>> >
>> > Thanks
>> > Ananth T Sarathy
>> >
>> >
>> > On Thu, Dec 3, 2009 at 1:55 PM, Jean-Daniel Cryans <jdcryans@apache.org
>> >wrote:
>> >
>> >> Could we see the region server's log? That errors is just about a
>> >> znode that already exists and that can happen and it's not fatal.
>> >>
>> >> J-D
>> >>
>> >> On Thu, Dec 3, 2009 at 10:52 AM, Ananth T. Sarathy
>> >> <an...@gmail.com> wrote:
>> >> > Hi,
>> >> > when my regionservers try to connect, I keep getting a zookeeper
>> >> timeout.
>> >> > on the server zookeeper is running i get this error on the zookeeper
>> log
>> >> >
>> >> > 2009-12-03 13:50:18,415 WARN
>> >> > org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>> >> > processing sessionid:0x12555e051b10001 type:create cxid:0x8
>> >> > zxid:0xfffffffffffffffe txntype:unknown n/a
>> >> > org.apache.zookeeper.KeeperException$NodeExistsException:
>> KeeperErrorCode
>> >> =
>> >> > NodeExists
>> >> > at
>> >> >
>> >>
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245)
>> >> > at
>> >> >
>> >>
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
>> >> >
>> >> > Any ideas on how to clear this up?
>> >> >
>> >> > Ananth T Sarathy
>> >> >
>> >>
>> >
>>
>
Re: Zookeeper problem
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
hmm ok. i ll try that. do I only need to change the
hbase.master.dns.nameserver and interface or should I override the zookeeper
and regionserver as well?
Ananth T Sarathy
On Thu, Dec 3, 2009 at 2:23 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Some problems I see:
>
> - ulimit - n 1024, the Getting Started documentation states that you
> should set it to a much higher number or you will run into problems at
> some point.
>
> - First time you start it says "java.net.NoRouteToHostException: No
> route to host" and the address it got is "192.168.20.89:60000". Seems
> like a network problem. This means the master reported it's address on
> 192.168.20.89.
>
> - Second time it tries to connect to the zookeeper server
> "192.169.20.89:2181" but gets a "Transport endpoint is not connected"
> so it cannot connect to it. The timeout you get is really a connection
> timeout while it retries to connect.
>
> So it seems your nodes are only able to connect when they use their
> hostname. One solution would be to play with
> hbase.master.dns.interface and hbase.master.dns.nameserver (see
> hbase-default.xml) and to set the quorum address to the hostname.
>
> J-D
>
> On Thu, Dec 3, 2009 at 11:13 AM, Ananth T. Sarathy
> <an...@gmail.com> wrote:
> > Sure
> >
> > http://pastebin.com/m670c216b
> >
> > Thanks
> > Ananth T Sarathy
> >
> >
> > On Thu, Dec 3, 2009 at 1:55 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
> >
> >> Could we see the region server's log? That errors is just about a
> >> znode that already exists and that can happen and it's not fatal.
> >>
> >> J-D
> >>
> >> On Thu, Dec 3, 2009 at 10:52 AM, Ananth T. Sarathy
> >> <an...@gmail.com> wrote:
> >> > Hi,
> >> > when my regionservers try to connect, I keep getting a zookeeper
> >> timeout.
> >> > on the server zookeeper is running i get this error on the zookeeper
> log
> >> >
> >> > 2009-12-03 13:50:18,415 WARN
> >> > org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
> >> > processing sessionid:0x12555e051b10001 type:create cxid:0x8
> >> > zxid:0xfffffffffffffffe txntype:unknown n/a
> >> > org.apache.zookeeper.KeeperException$NodeExistsException:
> KeeperErrorCode
> >> =
> >> > NodeExists
> >> > at
> >> >
> >>
> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245)
> >> > at
> >> >
> >>
> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
> >> >
> >> > Any ideas on how to clear this up?
> >> >
> >> > Ananth T Sarathy
> >> >
> >>
> >
>
Re: Zookeeper problem
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Some problems I see:
- ulimit - n 1024, the Getting Started documentation states that you
should set it to a much higher number or you will run into problems at
some point.
- First time you start it says "java.net.NoRouteToHostException: No
route to host" and the address it got is "192.168.20.89:60000". Seems
like a network problem. This means the master reported it's address on
192.168.20.89.
- Second time it tries to connect to the zookeeper server
"192.169.20.89:2181" but gets a "Transport endpoint is not connected"
so it cannot connect to it. The timeout you get is really a connection
timeout while it retries to connect.
So it seems your nodes are only able to connect when they use their
hostname. One solution would be to play with
hbase.master.dns.interface and hbase.master.dns.nameserver (see
hbase-default.xml) and to set the quorum address to the hostname.
J-D
On Thu, Dec 3, 2009 at 11:13 AM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> Sure
>
> http://pastebin.com/m670c216b
>
> Thanks
> Ananth T Sarathy
>
>
> On Thu, Dec 3, 2009 at 1:55 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> Could we see the region server's log? That errors is just about a
>> znode that already exists and that can happen and it's not fatal.
>>
>> J-D
>>
>> On Thu, Dec 3, 2009 at 10:52 AM, Ananth T. Sarathy
>> <an...@gmail.com> wrote:
>> > Hi,
>> > when my regionservers try to connect, I keep getting a zookeeper
>> timeout.
>> > on the server zookeeper is running i get this error on the zookeeper log
>> >
>> > 2009-12-03 13:50:18,415 WARN
>> > org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>> > processing sessionid:0x12555e051b10001 type:create cxid:0x8
>> > zxid:0xfffffffffffffffe txntype:unknown n/a
>> > org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
>> =
>> > NodeExists
>> > at
>> >
>> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245)
>> > at
>> >
>> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
>> >
>> > Any ideas on how to clear this up?
>> >
>> > Ananth T Sarathy
>> >
>>
>
Re: Zookeeper problem
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
Sure
http://pastebin.com/m670c216b
Thanks
Ananth T Sarathy
On Thu, Dec 3, 2009 at 1:55 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Could we see the region server's log? That errors is just about a
> znode that already exists and that can happen and it's not fatal.
>
> J-D
>
> On Thu, Dec 3, 2009 at 10:52 AM, Ananth T. Sarathy
> <an...@gmail.com> wrote:
> > Hi,
> > when my regionservers try to connect, I keep getting a zookeeper
> timeout.
> > on the server zookeeper is running i get this error on the zookeeper log
> >
> > 2009-12-03 13:50:18,415 WARN
> > org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
> > processing sessionid:0x12555e051b10001 type:create cxid:0x8
> > zxid:0xfffffffffffffffe txntype:unknown n/a
> > org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
> =
> > NodeExists
> > at
> >
> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245)
> > at
> >
> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
> >
> > Any ideas on how to clear this up?
> >
> > Ananth T Sarathy
> >
>
Re: Zookeeper problem
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Could we see the region server's log? That errors is just about a
znode that already exists and that can happen and it's not fatal.
J-D
On Thu, Dec 3, 2009 at 10:52 AM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> Hi,
> when my regionservers try to connect, I keep getting a zookeeper timeout.
> on the server zookeeper is running i get this error on the zookeeper log
>
> 2009-12-03 13:50:18,415 WARN
> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
> processing sessionid:0x12555e051b10001 type:create cxid:0x8
> zxid:0xfffffffffffffffe txntype:unknown n/a
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists
> at
> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245)
> at
> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
>
> Any ideas on how to clear this up?
>
> Ananth T Sarathy
>