You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jason Sheldon <ja...@synthasite.com> on 2008/05/09 15:49:55 UTC

Hbase error

Hi

I am very new to hbase and have just set it up for the first time. I  
have a running hadoop cluster with 5 datanodes and have installed  
HRegionServers on each of these nodes.

When I start up hbase I see this error in the master log:-

2008-05-09 15:43:35,922 INFO org.apache.hadoop.ipc.Server: IPC Server  
handler 7 on 60000, call regionServerStartup(address: 127.0.1.1:60020,  
startcode: 1210304455421, load: (requests: 0 regions: 0)) from  
10.211.55.4:55809: error: java.io.IOException:  
java.lang.AssertionError: Impossible state for createLease(): Lease  
1377822776/1377822776 is still held.
java.io.IOException: java.lang.AssertionError: Impossible state for  
createLease(): Lease 1377822776/1377822776 is still held.
	at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
	at org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
1310)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at  
sun 
.reflect 
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 
25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)


in the regionservers I see

2008-05-09 15:45:05,655 WARN org.apache.hadoop.hbase.HRegionServer:  
error telling master we are up
org.apache.hadoop.ipc.RemoteException: java.io.IOException:  
java.lang.AssertionError: Impossible state for createLease(): Lease  
1377822776/1377822776 is still held.
	at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
	at org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
1310)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at  
sun 
.reflect 
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 
25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)

	at org.apache.hadoop.ipc.Client.call(Client.java:512)
	at org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java: 
210)
	at $Proxy0.regionServerStartup(Unknown Source)
	at  
org.apache.hadoop.hbase.HRegionServer.reportForDuty(HRegionServer.java: 
1037)
	at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java:634)
	at java.lang.Thread.run(Thread.java:595)


As such I only see the master (localhost) region server as being active

HBase Version	0.1.1, r	HBase version and svn revision
Hadoop Version	0.16.2, r642481	Hadoop version and svn revision
Filesystem	hdfs://10.211.55.7:9000/hbase	Filesystem HBase is running on
HBase Root Directory	hdfs://10.211.55.7:9000/hbase	Location of HBase  
home directory
Online META Regions
Name	Server
-ROOT-	127.0.1.1:60020
.META.,,1	127.0.1.1:60020
Tables

No tables found.
Region Servers
	Address	Start Code	Load
127.0.1.1:60020	1210340213164	requests: 0 regions: 2
Total: 	servers: 1	 	requests: 0 regions: 2



Any explanation/help would be greatly appreciated

Thanks

Re: Hbase error

Posted by Andrew Purtell <ap...@yahoo.com>.
Hi.

You need to figure out why your region servers are all reporting with the
loopback address (127.0.0.1).

I encountered this problem before and in my case the cause was bogus
entries in /etc/hosts: The OS installer (I use RedHat Linux) would write
an entry into /etc/hosts that maps the configured host name to 127.0.0.1.
Why the RedHat OS installer (and perhaps others) might do this is a
debatable point. In my opinion, it should not. 

If something similar to this has happened in your environment, all of
your region servers -- regardless of whatever IP addresses might be
assigned to the Ethernet network interfaces and whatever hostname<->IP
address mappings you may have set up in your local DNS zone -- will be
reporting to the master with the same apparent identity (127.0.0.1).

So definitely you should look at your networking configuration and you
should start with /etc/hosts.

Hope this helps,

   - Andy

--- Jason Sheldon <ja...@synthasite.com> wrote:

> Hi
> 
> Thanks for the feedback. I got a little further but now it tells me:-
> 
> 2008-05-12 09:02:43,889 INFO org.apache.hadoop.ipc.Server: IPC Server  
> handler 1 on 60000, call regionServerStartup(address: 127.0.1.1:60020, 
> 
> startcode: 1210352247220, load: (requests: 0 regions: 0)) from  
> 10.211.55.6:51202: error: org.apache.hadoop.hbase.Leases 
> $LeaseStillHeldException
> org.apache.hadoop.hbase.Leases$LeaseStillHeldException
> 	at org.apache.hadoop.hbase.Leases.createLease(Leases.java:147)
> 	at org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
> 1311)
> 	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> 	at  
> sun 
> .reflect 
> .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
> 
> 25)
> 	at java.lang.reflect.Method.invoke(Method.java:585)
> 	at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
> 
> I have wiped the /hbase directory and restarted and it does not seem  
> to help
> 
> On 9 May 2008, at 18:12, stack wrote:
> 
> > Hey Jason.  Thanks for trying hbase.
> >
> > Can you take a look at your networking?  Remote regionservers are  
> > reporting in as 127.0.1.1:60020 rather than as 10.211.55.X.
> >
> > The master keeps an in-memory Map of all regionservers.  The key is  
> > the regionserver's reported Address+Port.  If all are reporting in  
> > as 127.0.1.1:60020, then the master is confused (AssertionError).
> >
> > Yours,
> > St.Ack
> >
> > P.S. Would suggest you also use the latest release candidate:
> http://people.apache.org/~stack/hbase-0.1.2-candidate-1/ 
> > .   It is much improved over 0.1.1.
> >
> >
> > Jason Sheldon wrote:
> >> Hi
> >>
> >> I am very new to hbase and have just set it up for the first time.  
> >> I have a running hadoop cluster with 5 datanodes and have installed 
> 
> >> HRegionServers on each of these nodes.
> >>
> >> When I start up hbase I see this error in the master log:-
> >>
> >> 2008-05-09 15:43:35,922 INFO org.apache.hadoop.ipc.Server: IPC  
> >> Server handler 7 on 60000, call regionServerStartup(address:  
> >> 127.0.1.1:60020, startcode: 1210304455421, load: (requests: 0  
> >> regions: 0)) from 10.211.55.4:55809: error: java.io.IOException:  
> >> java.lang.AssertionError: Impossible state for createLease(): Lease 
> 
> >> 1377822776/1377822776 is still held.
> >> java.io.IOException: java.lang.AssertionError: Impossible state for 
> 
> >> createLease(): Lease 1377822776/1377822776 is still held.
> >>    at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
> >>    at  
> >> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
> >> 1310)
> >>    at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> >>    at  
> >> sun 
> >> .reflect 
> >> .DelegatingMethodAccessorImpl 
> >> .invoke(DelegatingMethodAccessorImpl.java:25)
> >>    at java.lang.reflect.Method.invoke(Method.java:585)
> >>    at org.apache.hadoop.hbase.ipc.HbaseRPC 
> >> $Server.call(HbaseRPC.java:413)
> >>    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> >>
> >>
> >> in the regionservers I see
> >>
> >> 2008-05-09 15:45:05,655 WARN org.apache.hadoop.hbase.HRegionServer: 
> 
> >> error telling master we are up
> >> org.apache.hadoop.ipc.RemoteException: java.io.IOException:  
> >> java.lang.AssertionError: Impossible state for createLease(): Lease 
> 
> >> 1377822776/1377822776 is still held.
> >>    at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
> >>    at  
> >> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
> >> 1310)
> >>    at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> >>    at  
> >> sun 
> >> .reflect 
> >> .DelegatingMethodAccessorImpl 
> >> .invoke(DelegatingMethodAccessorImpl.java:25)
> >>    at java.lang.reflect.Method.invoke(Method.java:585)
> >>    at org.apache.hadoop.hbase.ipc.HbaseRPC 
> >> $Server.call(HbaseRPC.java:413)
> >>    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> >>
> >>    at org.apache.hadoop.ipc.Client.call(Client.java:512)
> >>    at org.apache.hadoop.hbase.ipc.HbaseRPC 
> >> $Invoker.invoke(HbaseRPC.java:210)
> >>    at $Proxy0.regionServerStartup(Unknown Source)
> >>    at  
> >> org 
> >> .apache.hadoop.hbase.HRegionServer.reportForDuty(HRegionServer.java:
> 
> >> 1037)
> >>    at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java: 
> >> 634)
> >>    at java.lang.Thread.run(Thread.java:595)
> >>
> >>
> >> As such I only see the master (localhost) region server as being  
> >> active
> >>
> >> HBase Version    0.1.1, r    HBase version and svn revision
> >> Hadoop Version    0.16.2, r642481    Hadoop version and svn revision
> >> Filesystem    hdfs://10.211.55.7:9000/hbase    Filesystem HBase is  
> >> running on
> >> HBase Root Directory    hdfs://10.211.55.7:9000/hbase    Location  
> >> of HBase home directory
> >> Online META Regions
> >> Name    Server
> >> -ROOT-    127.0.1.1:60020
> >> .META.,,1    127.0.1.1:60020
> >> Tables
> >>
> >> No tables found.
> >> Region Servers
> >>    Address    Start Code    Load
> >> 127.0.1.1:60020    1210340213164    requests: 0 regions: 2
> >> Total:     servers: 1         requests: 0 regions: 2
> >>
> >>
> >>
> >> Any explanation/help would be greatly appreciated
> >>
> >> Thanks
> >
> 
> Jason Sheldon
> Platform Engineer
> SynthaSite
> 
> Web: http://www.synthasite.com
> 
> 
> 
> 



      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Hbase error

Posted by Jason Sheldon <ja...@synthasite.com>.
Hi

Thanks for the feedback. I got a little further but now it tells me:-

2008-05-12 09:02:43,889 INFO org.apache.hadoop.ipc.Server: IPC Server  
handler 1 on 60000, call regionServerStartup(address: 127.0.1.1:60020,  
startcode: 1210352247220, load: (requests: 0 regions: 0)) from  
10.211.55.6:51202: error: org.apache.hadoop.hbase.Leases 
$LeaseStillHeldException
org.apache.hadoop.hbase.Leases$LeaseStillHeldException
	at org.apache.hadoop.hbase.Leases.createLease(Leases.java:147)
	at org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
1311)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at  
sun 
.reflect 
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 
25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)

I have wiped the /hbase directory and restarted and it does not seem  
to help

On 9 May 2008, at 18:12, stack wrote:

> Hey Jason.  Thanks for trying hbase.
>
> Can you take a look at your networking?  Remote regionservers are  
> reporting in as 127.0.1.1:60020 rather than as 10.211.55.X.
>
> The master keeps an in-memory Map of all regionservers.  The key is  
> the regionserver's reported Address+Port.  If all are reporting in  
> as 127.0.1.1:60020, then the master is confused (AssertionError).
>
> Yours,
> St.Ack
>
> P.S. Would suggest you also use the latest release candidate: http://people.apache.org/~stack/hbase-0.1.2-candidate-1/ 
> .   It is much improved over 0.1.1.
>
>
> Jason Sheldon wrote:
>> Hi
>>
>> I am very new to hbase and have just set it up for the first time.  
>> I have a running hadoop cluster with 5 datanodes and have installed  
>> HRegionServers on each of these nodes.
>>
>> When I start up hbase I see this error in the master log:-
>>
>> 2008-05-09 15:43:35,922 INFO org.apache.hadoop.ipc.Server: IPC  
>> Server handler 7 on 60000, call regionServerStartup(address:  
>> 127.0.1.1:60020, startcode: 1210304455421, load: (requests: 0  
>> regions: 0)) from 10.211.55.4:55809: error: java.io.IOException:  
>> java.lang.AssertionError: Impossible state for createLease(): Lease  
>> 1377822776/1377822776 is still held.
>> java.io.IOException: java.lang.AssertionError: Impossible state for  
>> createLease(): Lease 1377822776/1377822776 is still held.
>>    at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
>>    at  
>> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
>> 1310)
>>    at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>>    at  
>> sun 
>> .reflect 
>> .DelegatingMethodAccessorImpl 
>> .invoke(DelegatingMethodAccessorImpl.java:25)
>>    at java.lang.reflect.Method.invoke(Method.java:585)
>>    at org.apache.hadoop.hbase.ipc.HbaseRPC 
>> $Server.call(HbaseRPC.java:413)
>>    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
>>
>>
>> in the regionservers I see
>>
>> 2008-05-09 15:45:05,655 WARN org.apache.hadoop.hbase.HRegionServer:  
>> error telling master we are up
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException:  
>> java.lang.AssertionError: Impossible state for createLease(): Lease  
>> 1377822776/1377822776 is still held.
>>    at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
>>    at  
>> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: 
>> 1310)
>>    at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>>    at  
>> sun 
>> .reflect 
>> .DelegatingMethodAccessorImpl 
>> .invoke(DelegatingMethodAccessorImpl.java:25)
>>    at java.lang.reflect.Method.invoke(Method.java:585)
>>    at org.apache.hadoop.hbase.ipc.HbaseRPC 
>> $Server.call(HbaseRPC.java:413)
>>    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
>>
>>    at org.apache.hadoop.ipc.Client.call(Client.java:512)
>>    at org.apache.hadoop.hbase.ipc.HbaseRPC 
>> $Invoker.invoke(HbaseRPC.java:210)
>>    at $Proxy0.regionServerStartup(Unknown Source)
>>    at  
>> org 
>> .apache.hadoop.hbase.HRegionServer.reportForDuty(HRegionServer.java: 
>> 1037)
>>    at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java: 
>> 634)
>>    at java.lang.Thread.run(Thread.java:595)
>>
>>
>> As such I only see the master (localhost) region server as being  
>> active
>>
>> HBase Version    0.1.1, r    HBase version and svn revision
>> Hadoop Version    0.16.2, r642481    Hadoop version and svn revision
>> Filesystem    hdfs://10.211.55.7:9000/hbase    Filesystem HBase is  
>> running on
>> HBase Root Directory    hdfs://10.211.55.7:9000/hbase    Location  
>> of HBase home directory
>> Online META Regions
>> Name    Server
>> -ROOT-    127.0.1.1:60020
>> .META.,,1    127.0.1.1:60020
>> Tables
>>
>> No tables found.
>> Region Servers
>>    Address    Start Code    Load
>> 127.0.1.1:60020    1210340213164    requests: 0 regions: 2
>> Total:     servers: 1         requests: 0 regions: 2
>>
>>
>>
>> Any explanation/help would be greatly appreciated
>>
>> Thanks
>

Jason Sheldon
Platform Engineer
SynthaSite

Web: http://www.synthasite.com




Re: Hbase error

Posted by Andrew Purtell <ap...@yahoo.com>.
I encountered this problem as well. The RedHat/Centos installer (and
possibly others) adds an entry in /etc/hosts that points whatever host
name supplied during installation to 127.0.0.1. 

First, remove the bogus hostfile entry. Then, set up your local DNS to
serve the host names of your cluster hosts to your cluster. Then you
should be fine.

   - Andy

--- stack <st...@duboce.net> wrote:

> Hey Jason.  Thanks for trying hbase.
> 
> Can you take a look at your networking?  Remote regionservers are 
> reporting in as 127.0.1.1:60020 rather than as 10.211.55.X.
> 
> The master keeps an in-memory Map of all regionservers.  The key is the
> 
> regionserver's reported Address+Port.  If all are reporting in as 
> 127.0.1.1:60020, then the master is confused (AssertionError).
> 
> Yours,
> St.Ack
> 
> P.S. Would suggest you also use the latest release candidate: 
> http://people.apache.org/~stack/hbase-0.1.2-candidate-1/.   It is much 
> improved over 0.1.1.
> 
> 
> Jason Sheldon wrote:
> > Hi
> >
> > I am very new to hbase and have just set it up for the first time. I 
> > have a running hadoop cluster with 5 datanodes and have installed 
> > HRegionServers on each of these nodes.
> >
> > When I start up hbase I see this error in the master log:-
> >
> > 2008-05-09 15:43:35,922 INFO org.apache.hadoop.ipc.Server: IPC Server
> 
> > handler 7 on 60000, call regionServerStartup(address:
> 127.0.1.1:60020, 
> > startcode: 1210304455421, load: (requests: 0 regions: 0)) from 
> > 10.211.55.4:55809: error: java.io.IOException: 
> > java.lang.AssertionError: Impossible state for createLease(): Lease 
> > 1377822776/1377822776 is still held.
> > java.io.IOException: java.lang.AssertionError: Impossible state for 
> > createLease(): Lease 1377822776/1377822776 is still held.
> >     at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
> >     at 
> >
> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java:1310)
> >     at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> >     at 
> >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 
> >
> >     at java.lang.reflect.Method.invoke(Method.java:585)
> >     at 
> > org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
> >     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> >
> >
> > in the regionservers I see
> >
> > 2008-05-09 15:45:05,655 WARN org.apache.hadoop.hbase.HRegionServer: 
> > error telling master we are up
> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: 
> > java.lang.AssertionError: Impossible state for createLease(): Lease 
> > 1377822776/1377822776 is still held.
> >     at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
> >     at 
> >
> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java:1310)
> >     at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> >     at 
> >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 
> >
> >     at java.lang.reflect.Method.invoke(Method.java:585)
> >     at 
> > org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
> >     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
> >
> >     at org.apache.hadoop.ipc.Client.call(Client.java:512)
> >     at 
> >
> org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:210)
> >     at $Proxy0.regionServerStartup(Unknown Source)
> >     at 
> >
>
org.apache.hadoop.hbase.HRegionServer.reportForDuty(HRegionServer.java:1037)
> 
> >
> >     at
> org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java:634)
> >     at java.lang.Thread.run(Thread.java:595)
> >
> >
> > As such I only see the master (localhost) region server as being
> active
> >
> > HBase Version    0.1.1, r    HBase version and svn revision
> > Hadoop Version    0.16.2, r642481    Hadoop version and svn revision
> > Filesystem    hdfs://10.211.55.7:9000/hbase    Filesystem HBase is 
> > running on
> > HBase Root Directory    hdfs://10.211.55.7:9000/hbase    Location of 
> > HBase home directory
> > Online META Regions
> > Name    Server
> > -ROOT-    127.0.1.1:60020
> > .META.,,1    127.0.1.1:60020
> > Tables
> >
> > No tables found.
> > Region Servers
> >     Address    Start Code    Load
> > 127.0.1.1:60020    1210340213164    requests: 0 regions: 2
> > Total:     servers: 1         requests: 0 regions: 2
> >
> >
> >
> > Any explanation/help would be greatly appreciated
> >
> > Thanks
> 
> 



      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Hbase error

Posted by stack <st...@duboce.net>.
Hey Jason.  Thanks for trying hbase.

Can you take a look at your networking?  Remote regionservers are 
reporting in as 127.0.1.1:60020 rather than as 10.211.55.X.

The master keeps an in-memory Map of all regionservers.  The key is the 
regionserver's reported Address+Port.  If all are reporting in as 
127.0.1.1:60020, then the master is confused (AssertionError).

Yours,
St.Ack

P.S. Would suggest you also use the latest release candidate: 
http://people.apache.org/~stack/hbase-0.1.2-candidate-1/.   It is much 
improved over 0.1.1.


Jason Sheldon wrote:
> Hi
>
> I am very new to hbase and have just set it up for the first time. I 
> have a running hadoop cluster with 5 datanodes and have installed 
> HRegionServers on each of these nodes.
>
> When I start up hbase I see this error in the master log:-
>
> 2008-05-09 15:43:35,922 INFO org.apache.hadoop.ipc.Server: IPC Server 
> handler 7 on 60000, call regionServerStartup(address: 127.0.1.1:60020, 
> startcode: 1210304455421, load: (requests: 0 regions: 0)) from 
> 10.211.55.4:55809: error: java.io.IOException: 
> java.lang.AssertionError: Impossible state for createLease(): Lease 
> 1377822776/1377822776 is still held.
> java.io.IOException: java.lang.AssertionError: Impossible state for 
> createLease(): Lease 1377822776/1377822776 is still held.
>     at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
>     at 
> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java:1310)
>     at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>
>     at java.lang.reflect.Method.invoke(Method.java:585)
>     at 
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
>     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
>
>
> in the regionservers I see
>
> 2008-05-09 15:45:05,655 WARN org.apache.hadoop.hbase.HRegionServer: 
> error telling master we are up
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: 
> java.lang.AssertionError: Impossible state for createLease(): Lease 
> 1377822776/1377822776 is still held.
>     at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145)
>     at 
> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java:1310)
>     at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>
>     at java.lang.reflect.Method.invoke(Method.java:585)
>     at 
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
>     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
>
>     at org.apache.hadoop.ipc.Client.call(Client.java:512)
>     at 
> org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:210)
>     at $Proxy0.regionServerStartup(Unknown Source)
>     at 
> org.apache.hadoop.hbase.HRegionServer.reportForDuty(HRegionServer.java:1037) 
>
>     at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java:634)
>     at java.lang.Thread.run(Thread.java:595)
>
>
> As such I only see the master (localhost) region server as being active
>
> HBase Version    0.1.1, r    HBase version and svn revision
> Hadoop Version    0.16.2, r642481    Hadoop version and svn revision
> Filesystem    hdfs://10.211.55.7:9000/hbase    Filesystem HBase is 
> running on
> HBase Root Directory    hdfs://10.211.55.7:9000/hbase    Location of 
> HBase home directory
> Online META Regions
> Name    Server
> -ROOT-    127.0.1.1:60020
> .META.,,1    127.0.1.1:60020
> Tables
>
> No tables found.
> Region Servers
>     Address    Start Code    Load
> 127.0.1.1:60020    1210340213164    requests: 0 regions: 2
> Total:     servers: 1         requests: 0 regions: 2
>
>
>
> Any explanation/help would be greatly appreciated
>
> Thanks