You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by y_...@tsmc.com on 2010/01/27 04:09:53 UTC
RetriesExhaustedException
Hi,
Sorry for my wrong subject of prvious mail.
There are 3 zookeeper servers(4 core, 12G RAM), 10 Region servers (4 core,
12G RAM) in my cluster.
900 regions are in HBase now.
I don't know why the RetriesExhaustedException raised.
Any suggestions?
org.apache.hadoop.hbase.client.RetriesExhaustedException:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
region server Some server, retryOnlyOne=true, index=0, islastrow=true,
tries=9, numtries=10, i=19, listsize=20,
region=FRPF-MAINPD_ID,,1264583412962 for region
FRPF-MAINPD_ID,,1264583412962, row 'XRD_AE32739-01.001036q5gak_kd3gxd', but
failed after 10 attempts.
Exceptions:
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers$Batch.process(HConnectionManager.java:1120)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:1201)
at
org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:605)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:470)
at
org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.updateIndex(IndexedRegion.java:213)
at
org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.updateIndexes(IndexedRegion.java:145)
at
org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.put(IndexedRegion.java:96)
at
org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1208)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.put(HRegionServer.java:1834)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers$2.doCall(HConnectionManager.java:1193)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers$Batch.process(HConnectionManager.java:1115)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:1201)
at
org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:605)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:489)
at
flow.service.CommonTableData.insertToHbase(CommonTableData.java:287)
at flow.service.CommonTableData.insert(CommonTableData.java:61)
at CommonTable.main(CommonTable.java:23)
Fleming Chiu(邱宏明)
707-6128
y_823910@tsmc.com
週一無肉日吃素救地球(Meat Free Monday Taiwan)
---------------------------------------------------------------------------
TSMC PROPERTY
This email communication (and any attachments) is proprietary information
for the sole use of its
intended recipient. Any unauthorized review, use or distribution by anyone
other than the intended
recipient is strictly prohibited. If you are not the intended recipient,
please notify the sender by
replying to this email, and then delete this email and any copies of it
immediately. Thank you.
---------------------------------------------------------------------------
---------------------------------------------------------------------------
TSMC PROPERTY
This email communication (and any attachments) is proprietary information
for the sole use of its
intended recipient. Any unauthorized review, use or distribution by anyone
other than the intended
recipient is strictly prohibited. If you are not the intended recipient,
please notify the sender by
replying to this email, and then delete this email and any copies of it
immediately. Thank you.
---------------------------------------------------------------------------
Re: RetriesExhaustedException
Posted by Stack <st...@duboce.net>.
2010/1/26 <y_...@tsmc.com>:
>
> There are 3 zookeeper servers(4 core, 12G RAM), 10 Region servers (4 core,
> 12G RAM) in my cluster.
Regards the zk servers, thats a bit of overkill. You might instead
run them beside regionservers only give them their own disk if you
can. They need a bit of RAM and then they need to be able to write to
disk w/ minimal contention.
> 900 regions are in HBase now.
> I don't know why the RetriesExhaustedException raised.
> Any suggestions?
>
> org.apache.hadoop.hbase.client.RetriesExhaustedException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> region server Some server, retryOnlyOne=true, index=0, islastrow=true,
> tries=9, numtries=10, i=19, listsize=20,
> region=FRPF-MAINPD_ID,,1264583412962 for region
> FRPF-MAINPD_ID,,1264583412962, row 'XRD_AE32739-01.001036q5gak_kd3gxd', but
> failed after 10 attempts.
> Exceptions:
>
So, take the region name:
FRPF-MAINPD_ID,,1264583412962
...and grep the master log. See what was going on around the above
time. Was the region not deployed anywhere? If so, why not?
Hopefully you have been running at DEBUG log level. If so, you'll
have more detail in the logs and it'll be easier discerning why the
issue.
St.Ack