Bunch of errros/exceptions in 2 node hbase cluster setup

[Vimal Jain <vk...@gmail.com>]

Hi,
I am configuring 2 node hbase cluster atop 2 node Hadoop cluster for
production.
I am getting below warning/error/info/fail/exceptions in
master,region,zookeeper logs.

Please help in debugging these.What each means ?
Master logs
1)
WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
/backup-masters/ip-10-14-24-19.ap-southeast-1.compute.internal,60000,1394103610599
already deleted, and this is not a retry

2)
ERROR org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics: Incons
istent configuration. Previous configuration for using table name in
metrics: true, new configuration: false

3)
 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
/root-region-server already deleted, and this is not a retry

WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
/table92/-ROOT- already deleted, and this is not a retry

WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
/table92/.META. already deleted, and this is not a retry

Region Server logs

1)
ERROR org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics:
Inconsistent configuration. Previous configuration for using table name in
metrics: true, new configuration: false

Zookeeper logs

1)
ERROR org.apache.zookeeper.server.quorum.QuorumPeerConfig: Invalid co
nfiguration, only one server specified (ignoring)

2)
INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
 KeeperException when processing sessionid:0x144970cd8560000 type:delete
cxid:0x14 zxid:0xc txntype:-1 reqpath:n/a Error Path:/hbase/ba
ckup-masters/ip-10-14-24-19.ap-southeast-1.compute.internal,60000,1394103610599
Error:KeeperErrorCode = NoNode for /hbase/backup-master
s/ip-10-14-24-19.ap-southeast-1.compute.internal,60000,1394103610599

(This is repeated 6-7 times ).
-- 
Thanks and Regards,
Vimal Jain

Re: Bunch of errros/exceptions in 2 node hbase cluster setup

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Comments below. I will skip most of the warnings as they are only warnings
and due to your setup. Will also skip duplicate messages.

>From what I can see, there is nothing to be really worried about in what
you sent.

JMS


2014-03-06 6:10 GMT-05:00 Vimal Jain <vk...@gmail.com>:

> Hi,
> I am configuring 2 node hbase cluster atop 2 node Hadoop cluster for
> production.
> I am getting below warning/error/info/fail/exceptions in
> master,region,zookeeper logs.
>

Since recommended minimum replication in hadoop is 3, you will not be able
to achieve this with only 2 nodes.



> Please help in debugging these.What each means ?
> Master logs
> 1)
> WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
>
> /backup-masters/ip-10-14-24-19.ap-southeast-1.compute.internal,60000,1394103610599
> already deleted, and this is not a retry
>
> 2)
> ERROR org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics: Incons
> istent configuration. Previous configuration for using table name in
> metrics: true, new configuration: false
>

I have seen this one many times. I don't think it shoule be with ERROR log
level. More probably INFO or maybe WARN.


>
> 3)
>  WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
> /root-region-server already deleted, and this is not a retry
>
> WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
> /table92/-ROOT- already deleted, and this is not a retry
>
> WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase
> /table92/.META. already deleted, and this is not a retry
>
> Region Server logs
>
> 1)
> ERROR org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics:
> Inconsistent configuration. Previous configuration for using table name in
> metrics: true, new configuration: false
>
> Zookeeper logs
>
> 1)
> ERROR org.apache.zookeeper.server.quorum.QuorumPeerConfig: Invalid co
> nfiguration, only one server specified (ignoring)
>

You have only one ZK server. That's most probably why youget this message.
To have a Quorum, you need to have 3 or 5 ZK servers.


>
> 2)
> INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
>  KeeperException when processing sessionid:0x144970cd8560000 type:delete
> cxid:0x14 zxid:0xc txntype:-1 reqpath:n/a Error Path:/hbase/ba
>
> ckup-masters/ip-10-14-24-19.ap-southeast-1.compute.internal,60000,1394103610599
> Error:KeeperErrorCode = NoNode for /hbase/backup-master
> s/ip-10-14-24-19.ap-southeast-1.compute.internal,60000,1394103610599
>
> (This is repeated 6-7 times ).
> --
> Thanks and Regards,
> Vimal Jain
>