You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Gaojinchao <ga...@huawei.com> on 2011/04/19 05:43:01 UTC
A question about Hmaster restarted.
I created table with some regions.
Hmaster had crashed because of one region server crashed.
I dig the code. It may be a bug.
Startup or create table use this code.
In startup case need to shutdown itself.
But ,create table need to reassign.
long maxWaitTime = System.currentTimeMillis() +
this.master.getConfiguration().getLong("hbase.regionserver.rpc.startup.waittime", 60000);
while (!this.master.isStopped()) {
try {
this.serverManager.sendRegionOpen(destination, regions);
break;
} catch (org.apache.hadoop.hbase.ipc.ServerNotRunningException e) {
// This is the one exception to retry. For all else we should just fail
// the startup.
long now = System.currentTimeMillis();
if (now > maxWaitTime) throw e;
LOG.debug("Server is not yet up; waiting up to " +
(maxWaitTime - now) + "ms", e);
Thread.sleep(1000);
}
}
} catch (Throwable t) {
this.master.abort("Failed assignment of regions to " + destination +
"; bulk assign FAILED", t);
return;
}
Hmaster logs:
2011-04-18 20:21:37,144 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
2011-04-18 20:21:47,012 FATAL org.apache.hadoop.hbase.master.HMaster: Failed assignment of regions to serverName=t5,60020,1303129296185, load=(requests=0, regions=0, usedHeap=0, maxHeap=0)
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningException: Server is not running yet
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy6.openRegions(Unknown Source)
at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:566)
at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:776)
at org.apache.hadoop.hbase.master.AssignmentManager$SingleServerBulkAssigner.run(AssignmentManager.java:1310)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2011-04-18 20:21:47,012 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2011-04-18 20:21:47,012 FATAL org.apache.hadoop.hbase.master.HMaster: Failed assignment of regions to serverName=t5,60020,1303129296185, load=(requests=0, regions=0, usedHeap=0, maxHeap=0)
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningException: Server is not running yet
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038)
Re: A question about Hmaster restarted.
Posted by Jean-Daniel Cryans <jd...@apache.org>.
See HBASE-3744, createTable shouldn't be using the startup bulk assigner.
J-D
On Mon, Apr 18, 2011 at 8:43 PM, Gaojinchao <ga...@huawei.com> wrote:
> I created table with some regions.
> Hmaster had crashed because of one region server crashed.
>
> I dig the code. It may be a bug.
> Startup or create table use this code.
> In startup case need to shutdown itself.
> But ,create table need to reassign.
>
> long maxWaitTime = System.currentTimeMillis() +
> this.master.getConfiguration().getLong("hbase.regionserver.rpc.startup.waittime", 60000);
> while (!this.master.isStopped()) {
> try {
> this.serverManager.sendRegionOpen(destination, regions);
> break;
> } catch (org.apache.hadoop.hbase.ipc.ServerNotRunningException e) {
> // This is the one exception to retry. For all else we should just fail
> // the startup.
> long now = System.currentTimeMillis();
> if (now > maxWaitTime) throw e;
> LOG.debug("Server is not yet up; waiting up to " +
> (maxWaitTime - now) + "ms", e);
> Thread.sleep(1000);
> }
> }
> } catch (Throwable t) {
> this.master.abort("Failed assignment of regions to " + destination +
> "; bulk assign FAILED", t);
> return;
> }
>
> Hmaster logs:
> 2011-04-18 20:21:37,144 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
> 2011-04-18 20:21:47,012 FATAL org.apache.hadoop.hbase.master.HMaster: Failed assignment of regions to serverName=t5,60020,1303129296185, load=(requests=0, regions=0, usedHeap=0, maxHeap=0)
> org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningException: Server is not running yet
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038)
>
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
> at $Proxy6.openRegions(Unknown Source)
> at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:566)
> at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:776)
> at org.apache.hadoop.hbase.master.AssignmentManager$SingleServerBulkAssigner.run(AssignmentManager.java:1310)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2011-04-18 20:21:47,012 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
> 2011-04-18 20:21:47,012 FATAL org.apache.hadoop.hbase.master.HMaster: Failed assignment of regions to serverName=t5,60020,1303129296185, load=(requests=0, regions=0, usedHeap=0, maxHeap=0)
> org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningException: Server is not running yet
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038)
>
>