You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Ajay Govindarajan <ag...@yahoo.com> on 2011/04/07 07:45:30 UTC
HTable.put hangs on bulk loading
I am doing a load test for which I need to load a table with many rows.
I have a small java program that has a for loop and calls HTable.put.
I am inserting a map of 2 items into a table that has one column
family. The limit of the for loop is currently 20000. However after
15876 rows the call to Put hangs. I am using autoFlush on the HTable. I think its a problem with Execute threads in the java client.
Any ideas will be greatly appreciated.
The table configuration:
DESCRIPTION ENABLED
{NAME => 'TABLE2', FAMILIES => [{NAME => 'TABLE2_CF true
1', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0'
, COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2
147483647', BLOCKSIZE => '65536', IN_MEMORY => 'fal
se', BLOCKCACHE => 'true'}]}
The HBase config on the client is the one in the hbase-default.xml. Some values:
hbase.client.write.buffer=2097152
hbase.client.pause=1000
hbase.client.retries.number=10
If i use another client I am able to put items to the table. I am also able to scan items from the table using the hbase shell.
I don't see anything in the region server or master logs. I have them here.
The master server log:
2011-04-06 19:02:40,149 INFO
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 184.106.69.238:60020, regionname:
-ROOT-,,0.70236052, startKey: <>}
2011-04-06 19:02:40,152 INFO
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scan of 1 row(s) of meta region {server: 184.106.69.238:60020,
regionname: -ROOT-,,0.70236052, startKey: <>} complete
2011-04-06 19:02:40,157 INFO org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 0 dead, average load 42.0
2011-04-06 19:03:15,252 INFO
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner
scanning meta region {server: 184.106.69.238:60020, regionname:
.META.,,1.1028785192, startKey: <>}
2011-04-06 19:03:15,265 INFO
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner
scan of 40 row(s) of meta region {server: 184.106.69.238:60020,
regionname: .META.,,1.1028785192, startKey: <>} complete
2011-04-06 19:03:15,266 INFO org.apache.hadoop.hbase.master.BaseScanner: All 1 .META. region(s) scanned
The region server logs:
2011-04-06 19:02:21,294 DEBUG
org.apache.hadoop.hbase.regionserver.HRegion: Creating region
TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.
2011-04-06 19:02:21,295 INFO
org.apache.hadoop.hbase.regionserver.HRegion: Onlined
TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.; next
sequenceid=1