答复: 答复: 答复: 答复: Region has been CLOSING for too long, this should eventually complete or the server will expire, send RPC again

[dylan <dw...@gmail.com>]

Yes  I  set export HBASE_MANAGES_ZK=false in the hbase-env.sh

 

发件人: Azuryy Yu [mailto:azuryyyu@gmail.com] 
发送时间: 2013年4月16日 10:59
收件人: user@hadoop.apache.org
主题: Re: 答复: 答复: 答复: Region has been CLOSING for too long, this
should eventually complete or the server will expire, send RPC again

 

did your hbase managed zookeeper? or did you set export
HBASE_MANAGES_ZK=false in the hbase-env.sh?

 

if not, then that's zookeeper port conflicted.

 

On Tue, Apr 16, 2013 at 10:55 AM, dylan <dw...@gmail.com> wrote:

# The number of milliseconds of each tick

tickTime=2000

# The number of ticks that the initial 

# synchronization phase can take

initLimit=10

# The number of ticks that can pass between 

# sending a request and getting an acknowledgement

syncLimit=5

# the directory where the snapshot is stored.

# do not use /tmp for storage, /tmp here is just 

# example sakes.

dataDir=/usr/cdh4/zookeeper/data

# the port at which the clients will connect

clientPort=2181

 

server.1=Slave01:2888:3888

server.2=Slave02:2888:3888

server.3=Slave03:2888:3888

 

发件人: Azuryy Yu [mailto:azuryyyu@gmail.com] 

发送时间: 2013年4月16日 10:45
收件人: user@hadoop.apache.org
主题: Re: 答复: 答复: Region has been CLOSING for too long, this should
eventually complete or the server will expire, send RPC again

 

and paste ZK configuration in the zookeerp_home/conf/zoo.cfg

 

On Tue, Apr 16, 2013 at 10:42 AM, Azuryy Yu <az...@gmail.com> wrote:

it located under hbase-home/logs/  if your zookeeper is managed by hbase.

 

but I noticed you configured QJM, then did your QJM and Hbase share the same
ZK cluster? if so, then just paste your QJM zk configuration in the
hdfs-site.xml and hbase zk configuration in the hbase-site.xml.

 

On Tue, Apr 16, 2013 at 10:37 AM, dylan <dw...@gmail.com> wrote:


How to check zookeeper log?? It is the binary files, how to transform it to
normal log? 


 


I find the “org.apache.zookeeper.server.LogFormatter”, how to run?


 


 

发件人: Azuryy Yu [mailto:azuryyyu@gmail.com] 
发送时间: 2013年4月16日 10:01
收件人: user@hadoop.apache.org
主题: Re: 答复: Region has been CLOSING for too long, this should eventually
complete or the server will expire, send RPC again

 

This is zookeeper issue.

 

please paste zookeeper log here. thanks.

 

On Tue, Apr 16, 2013 at 9:58 AM, dylan <dw...@gmail.com> wrote:

It is hbase-0.94.2-cdh4.2.0.

 

发件人: Ted Yu [mailto:yuzhihong@gmail.com] 
发送时间: 2013年4月16日 9:55
收件人: user@hbase.apache.org
主题: Re: Region has been CLOSING for too long, this should eventually
complete or the server will expire, send RPC again

 

I think this question would be more appropriate for HBase user mailing list.

 

Moving hadoop user to bcc.

 

Please tell us the HBase version you are using.

 

Thanks

On Mon, Apr 15, 2013 at 6:51 PM, dylan <dw...@gmail.com> wrote:

Hi

 

I am a newer for hadoop, and set up hadoop with tarball . I have 5 nodes for
cluster, 2 NN nodes with QJM (3 Journal Nodes, one of them on DN node.  ), 3
DN nodes with zookeepers,  It works fine.  When I reboot one data node
machine which includes zookeeper, after that , restart all processes. The
hadoop works fine, but hbase not. I cannot disable tables and drop tables.

 

The logs an follows:

The Hbase HMaster log:

DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Attempted to
unassign region -ROOT-,,0.70236052 but it is not currently assigned anywhere

,683 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in
transition timed out:  -ROOT-,,0.70236052 state=CLOSING, ts=1366001558865,
server=Master,60000,1366001238313

,683 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been
CLOSING for too long, this should eventually complete or the server will
expire, send RPC again

10,684 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting
unassignment of region -ROOT-,,0.70236052 (offlining)

 

The Hbase HRegionServer log:

 

DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Stats: total=7.44 MB,
free=898.81 MB, max=906.24 MB, blocks=0, accesses=0, hits=0, hitRatio=0,
cachingAccesses=0, cachingHits=0, cachingHitsRatio=0, evictions=0,
evicted=0, evictedPerRun=NaN

 

The Hbase Web show：

Region                                              State

70236052    -ROOT-,,0.70236052 state=CLOSING, ts=Mon Apr 15 12:52:38 CST
2013 (75440s ago), server=Master,60000,1366001238313

 

How fix it?

 

Thanks.