Re: Hbase 1.0 Distributed Mode:Issue:Master is initializing

[Sean Busbey <bu...@cloudera.com>]

Hi Sridhar!

Please be sure to copy your responses to user@hbase and not just an
individual.

The attached log shows that your master service does not see any Region
Servers as available. You should check the logs for the region servers to
see why they aren't starting.

On Sat, Apr 4, 2015 at 11:37 AM, sridhararao mutluri <dr...@hotmail.com>
wrote:

> Sean,
>
> I used mycluster/hbase in hbase-site.xml and no luck to me.
>
> I tried to use 0.98.4 with distributed mode and hadoop ha.Now new error is
> coming:
>
>
> hbase(main):002:0> create 'cars', 'vi'
>
> ERROR: java.io.IOException: Table Namespace Manager not ready yet, try
> again later
>         at
> org.apache.hadoop.hbase.master.HMaster.getNamespaceDescriptor(HMaster.java:3179)
>         at
> org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1735)
>         at
> org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1774)
>         at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40470)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
>         at
> org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> ZK are running on 3 servers.I tired to stop hbase and then stopped zk in 3
> clusters and cleared zk data and started fresh.
>
> I tried to use only hostname instead of ip.
>
> I am attaching log for your personal observation and good suggestions.
>
> Thanks,
> Sridhar
>
>
> > From: busbey@cloudera.com
> > Date: Fri, 3 Apr 2015 11:34:24 -0500
>
> > Subject: Re: Hbase 1.0 Distributed Mode:Issue:Master is initializing
> > To: user@hbase.apache.org
> >
> > It should be
> >
> > <property>
> > <name>hbase.rootdir</name>
> > <value>hdfs://mycluster/hbase</value>
> > </property>
> >
> >
> > On Fri, Apr 3, 2015 at 11:19 AM, sridhararao mutluri <drmsr6@hotmail.com
> >
> > wrote:
> >
> > > Hi Sean,
> > > Thanks for your speedy reply.
> > > We are suing hadoop HA with namespace "mycluster".
> > > Please suggest below property value like this:
> > > <name>hbase.rootdir</name>
> > > <value>hdfs://mycluster:8020/hbase</value> </property>Thanks,Sridhar>
> > > From: busbey@cloudera.com
> > > > Date: Fri, 3 Apr 2015 11:11:42 -0500
> > > > Subject: Re: Hbase 1.0 Distributed Mode:Issue:Master is initializing
> > > > To: user@hbase.apache.org
> > > >
> > > > you should check the master's log for what is causing it to be
> delayed in
> > > > starting.
> > > >
> > > > A few configuration errors:
> > > >
> > > > 1) zk dataDir in hbase configs
> > > >
> > > > <property> <name>hbase.zookeeper.property.dataDir</name>
> > > > <value>/hadoop/hdfs/zookeeper/data/zk1</value> </property>
> > > >
> > > > This property is not needed, because you have specified a ZK quorum
> and
> > > are
> > > > in distributed mode.
> > > >
> > > > 2) HDFS address
> > > >
> > > > <property> <name>fs.defaultFS</name>
> > > > <value>hdfs://mycluster</value>
> > > > </property>
> > > >
> > > > Did you redact this, or is your HDFS set up in HA with the
> nameservice
> > > > "mycluster"?
> > > >
> > > > If it's the latter, then this config is wrong:
> > > >
> > > > <property> <name>hbase.rootdir</name>
> > > <value>hdfs://
> > > > 10.1.252.86:8020/hbase</value> </property>
> > > >
> > > > This is presumably the address of one of your NameNodes. If you are
> using
> > > > HA, then this should use the nameservice instead of a particular
> host.
> > > > Otherwise, HBase will get errors whenever the named host is not
> active.
> > > >
> > > > 3) IP addresses
> > > >
> > > > In general, you should be using FQDN for hosts in your cluster and
> not IP
> > > > addresses. If you are using IP addresses because DNS is problematic,
> > > please
> > > > be aware that several parts of the cluster are going to attempt to
> use
> > > > reverse DNS to turn that back into a host name. You can save yourself
> > > > trouble by doing the work upfront to make sure forward and reverse
> DNS
> > > > behaves properly (even if just with coordinated /etc/hosts files)
> > > >
> > > >
> > > > On Fri, Apr 3, 2015 at 11:04 AM, sridhararao mutluri <
> drmsr6@hotmail.com
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > >
> > > > >
> > > > > Team,
> > > > > I am facing below error.Please suggest solution:
> > > > > hbase(main):001:0> create 'cars', 'vi'
> > > > > ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is
> > > > > initializing at
> > > > >
> > >
> org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:1869)
> > > > > at
> > > > >
> > >
> org.apache.hadoop.hbase.master.HMaster.checkNamespaceManagerReady(HMaster.java:1874)
> > > > > at
> > > > >
> > >
> org.apache.hadoop.hbase.master.HMaster.ensureNamespaceExists(HMaster.java:2067)
> > > > > at
> > > > >
> org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1262)
> > > > > at
> > > > >
> > >
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> > > > > at
> > > > >
> > >
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> > > > > at
> > > org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> > > > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> > > > > at
> > > > >
> > >
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> > > > > at
> > > > > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > hbase-site.xml shows:
> > > > > <configuration> <property> <name>hbase.master</name>
> > > > > <value>10.1.252.86:60000</value> </property>
> > > <property>
> > > > > <name>hbase.rootdir</name> <value>hdfs://
> > > > > 10.1.252.86:8020/hbase</value> </property> <property>
> > > > > <name>hbase.zookeeper.property.dataDir</name>
> > > > > <value>/hadoop/hdfs/zookeeper/data/zk1</value> </property>
> > > > > <property> <name>hbase.cluster.distributed</name>
> > > > > <value>true</value> </property> <property>
> > > > > <name>hbase.zookeeper.property.clientPort</name>
> > > > > <value>2181</value> </property> <property>
> > > > > <name>hbase.zookeeper.quorum</name>
> > > > > <value>10.1.252.86,10.1.252.87,10.1.252.88</value> </property>
> > > > >
> > > > > Hadoop core-site.xml:
> > > > > <configuration> <property> <name>fs.defaultFS</name>
> > > > > <value>hdfs://mycluster</value> </property> <property>
> > > > > Zookeeper status:[hadoop@...... bin]$ ./zkServer.sh statusJMX
> enabled
> > > by
> > > > > defaultUsing config:
> > > /home/hadoop/zookeeper-3.4.6/bin/../conf/zoo.cfgMode:
> > > > > follower
> > > > > Thanks,Sridhar
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sean
> > >
> > >
> >
> >
> >
> > --
> > Sean
>



-- 
Sean