You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "lanfengcheng@xmtr" <la...@xmtongrong.com> on 2012/11/09 03:37:09 UTC
hbase slave region server is terminated unexpectedly
HI,
Recently our HBASE cluster meet a situation that one of our HRegionServer nodes (random one) stop unexpectedly .
our HBASE servers are in high concurrent write data and read and write data.
At first we thought it may be caused due to configuration problem, we therefore tried the optimization of HBASE parameter .
But it still happended after the optimization of the parameters, the process termination of HRegionServer.
Please help us and give some suggestions or solutions, thank you.
Environment:
My HBase culster contains 7 computers. One is the master and zookeeper server, the other 6 are the region servers.
The operatiing system is Centos 5.6 with kenerl 2.6.18-238.el5.
The jdk version is 1.7.0_03
HBASE VERSION:0.94.2
HADOOP VERSION:Hadoop 1.0.4
PARTS OF HBASE-SITE.XML CONFIGURATIONS:
----------------------------------------------------------------------
<property>
<name>hbase.regionserver.lease.period</name>
<value>180000</value>
</property>
<property>
<name>hbase.hregion.max.filesize</name>
<value>2048000000</value>
</property>
<property>
<name>hbase.hregion.memstore.flush.size</name>
<value>512000000</value>
</property>
<property>
<name>zookeeper.session.timeout</name>
<value>60000</value>
</property>
<property>
<name>hbase.regionserver.handler.count</name>
<value>50</value>
</property>
LOG BEGIN----------------------------------------------------------------------
REGION SERVER ERROR LOG:
2012-11-09 00:00:09,312 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:00:09,354 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.67 MB, single=154.11 MB, multi=463.94 MB, memory=0 KB
2012-11-09 00:00:27,770 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.43 MB
2012-11-09 00:00:27,781 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.45 MB, total=549.98 MB, single=152.15 MB, multi=465.26 MB, memory=0 KB
2012-11-09 00:00:45,869 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:00:45,880 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.08 MB, single=154.92 MB, multi=462.54 MB, memory=0 KB
2012-11-09 00:00:58,576 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 00:00:58,580 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3. due to global heap pressure
2012-11-09 00:00:58,580 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3., current region memstore size 104.2m
2012-11-09 00:00:58,580 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3., commencing wait for mvcc, flushsize=109299760
2012-11-09 00:00:58,580 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 00:00:58,625 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524with permission:rwxrwxrwx
2012-11-09 00:00:58,657 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:00:58,657 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524: CompoundBloomFilterWriter
2012-11-09 00:00:59,857 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524)
2012-11-09 00:00:59,857 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=248607, memsize=104.2m, into tmp file hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524
2012-11-09 00:00:59,864 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524 to hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/c7099a89751b4bedb3e9554569169524
2012-11-09 00:00:59,915 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/c7099a89751b4bedb3e9554569169524, entries=421945, sequenceid=248607, filesize=32.4m
2012-11-09 00:00:59,917 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~104.2m/109299760, currentsize=0.0/0 for region taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3. in 1336ms, sequenceid=248607, compaction requested=true
2012-11-09 00:00:59,917 DEBUG org.apache.hadoop.hbase.regionserver.Store: 9502c1830ad553d1e85497f9445d30d3 - redirect_info: Initiating minorcompaction
2012-11-09 00:00:59,917 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on redirect_info in region taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
2012-11-09 00:00:59,917 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small Compaction requested: regionName=taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3., storeName=redirect_info, fileCount=3, fileSize=270.7m (204.6m, 33.6m, 32.4m), priority=3, time=696695448849702; Because: regionserver60020.cacheFlusher; compaction_queue=(0:0), split_queue=0
2012-11-09 00:00:59,918 INFO org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3 file(s) in redirect_info of taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3. into tmpdir=hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp, seqid=248607, totalSize=270.7m
2012-11-09 00:00:59,918 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/95ea533c79b14d70b816136d79402235, keycount=2633480, bloomtype=NONE, size=204.6m, encoding=NONE
2012-11-09 00:00:59,918 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/9419b66fd8e34977a8b0f48b4aa1ec05, keycount=438245, bloomtype=NONE, size=33.6m, encoding=NONE
2012-11-09 00:00:59,918 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/c7099a89751b4bedb3e9554569169524, keycount=421945, bloomtype=NONE, size=32.4m, encoding=NONE
2012-11-09 00:00:59,955 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868ewith permission:rwxrwxrwx
2012-11-09 00:00:59,989 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:00:59,989 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868e: CompoundBloomFilterWriter
2012-11-09 00:01:04,115 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:01:04,126 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=549.99 MB, single=148.89 MB, multi=468.51 MB, memory=0 KB
2012-11-09 00:01:09,559 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868e)
2012-11-09 00:01:09,566 INFO org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868e to hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/ba824998e14e4a13a309cfa933a4868e
2012-11-09 00:01:09,752 INFO org.apache.hadoop.hbase.regionserver.Store: Completed compaction of 3 file(s) in redirect_info of taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3. into ba824998e14e4a13a309cfa933a4868e, size=270.7m; total size for store is 1.2g
2012-11-09 00:01:09,752 INFO org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: completed compaction: regionName=taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3., storeName=redirect_info, fileCount=3, fileSize=270.7m, priority=3, time=696695448849702; duration=9sec
2012-11-09 00:01:09,752 DEBUG org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: CompactSplitThread status: compaction_queue=(0:0), split_queue=0
2012-11-09 00:01:23,517 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.4 MB of total=623.43 MB
2012-11-09 00:01:23,528 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.09 MB, single=142.6 MB, multi=474.81 MB, memory=0 KB
2012-11-09 00:01:41,897 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:01:41,908 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=549.97 MB, single=160.06 MB, multi=457.34 MB, memory=0 KB
2012-11-09 00:02:01,739 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:02:01,749 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.25 MB, single=146.18 MB, multi=471.34 MB, memory=0 KB
2012-11-09 00:02:20,779 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:02:20,791 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.05 MB, single=158.69 MB, multi=458.66 MB, memory=0 KB
2012-11-09 00:02:42,214 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:02:42,225 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.09 MB, single=145.46 MB, multi=471.98 MB, memory=0 KB
2012-11-09 00:02:59,501 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:02:59,512 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.05 MB, single=151.19 MB, multi=466.23 MB, memory=0 KB
2012-11-09 00:03:26,921 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.43 MB
2012-11-09 00:03:26,931 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.05 MB, single=144.43 MB, multi=472.98 MB, memory=0 KB
2012-11-09 00:04:01,946 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:04:01,957 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.31 MB, single=152.27 MB, multi=465.16 MB, memory=0 KB
2012-11-09 00:04:24,838 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:04:24,848 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.01 MB, single=150.66 MB, multi=466.7 MB, memory=0 KB
2012-11-09 00:04:39,188 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=592.36 MB, free=141.01 MB, max=733.38 MB, blocks=9276, accesses=4770844427, hits=4766817449, hitRatio=99.91%, cachingAccesses=4770192126, cachingHits=4766793835, cachingHitsRatio=99.92%, evictions=2892, evicted=3389016, evictedPerRun=1171.85888671875
2012-11-09 00:04:46,619 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:04:46,624 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.03 MB, single=161.92 MB, multi=455.46 MB, memory=0 KB
2012-11-09 00:05:04,154 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:05:04,165 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.02 MB, single=143.41 MB, multi=473.96 MB, memory=0 KB
2012-11-09 00:05:20,916 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:05:20,927 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.04 MB, single=158.61 MB, multi=458.83 MB, memory=0 KB
2012-11-09 00:05:37,691 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:05:37,701 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=549.99 MB, single=149.13 MB, multi=468.24 MB, memory=0 KB
2012-11-09 00:05:59,474 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:05:59,485 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=550.23 MB, single=148.26 MB, multi=469.11 MB, memory=0 KB
2012-11-09 00:06:16,659 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.39 MB
2012-11-09 00:06:16,670 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=549.98 MB, single=157.47 MB, multi=459.9 MB, memory=0 KB
2012-11-09 00:06:34,145 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:06:34,152 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.09 MB, single=149.44 MB, multi=467.94 MB, memory=0 KB
2012-11-09 00:06:49,302 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:06:49,314 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.3 MB, single=151.87 MB, multi=465.61 MB, memory=0 KB
2012-11-09 00:07:08,734 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.43 MB of total=623.46 MB
2012-11-09 00:07:08,748 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.02 MB, single=146.79 MB, multi=470.65 MB, memory=0 KB
2012-11-09 00:07:28,549 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:07:28,560 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.43 MB, total=550 MB, single=146.44 MB, multi=470.97 MB, memory=0 KB
2012-11-09 00:07:47,367 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:07:47,372 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.13 MB, single=150.1 MB, multi=467.33 MB, memory=0 KB
2012-11-09 00:08:11,442 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:08:11,453 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=550.04 MB, single=148.84 MB, multi=468.53 MB, memory=0 KB
2012-11-09 00:08:31,151 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:08:31,162 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=550.08 MB, single=158.79 MB, multi=458.61 MB, memory=0 KB
2012-11-09 00:08:48,322 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 00:08:48,323 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. due to global heap pressure
2012-11-09 00:08:48,323 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6., current region memstore size 122.6m
2012-11-09 00:08:48,323 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6., commencing wait for mvcc, flushsize=128508424
2012-11-09 00:08:48,323 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 00:08:48,358 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940ewith permission:rwxrwxrwx
2012-11-09 00:08:48,400 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:08:48,400 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e: CompoundBloomFilterWriter
2012-11-09 00:08:49,977 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e)
2012-11-09 00:08:49,977 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=248709, memsize=122.6m, into tmp file hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e
2012-11-09 00:08:49,984 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e to hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/571dc1e866184ca6aed3baa2d491940e
2012-11-09 00:08:50,038 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/571dc1e866184ca6aed3baa2d491940e, entries=443292, sequenceid=248709, filesize=47.3m
2012-11-09 00:08:50,040 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~122.6m/128508424, currentsize=0.0/0 for region taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. in 1717ms, sequenceid=248709, compaction requested=true
2012-11-09 00:08:50,040 DEBUG org.apache.hadoop.hbase.regionserver.Store: 77c58ab3b4586478b84f4922950d08e6 - record_info: Initiating minorcompaction
2012-11-09 00:08:50,040 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small Compaction requested: regionName=taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6., storeName=record_info, fileCount=3, fileSize=133.6m (53.8m, 32.5m, 47.3m), priority=2, time=697165571949702; Because: regionserver60020.cacheFlusher; compaction_queue=(0:1), split_queue=0
2012-11-09 00:08:50,040 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on record_info in region taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
2012-11-09 00:08:50,040 INFO org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3 file(s) in record_info of taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. into tmpdir=hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp, seqid=248709, totalSize=133.6m
2012-11-09 00:08:50,040 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/836efce1f8944c79b2cad46cd70bcfd4, keycount=507310, bloomtype=NONE, size=53.8m, encoding=NONE
2012-11-09 00:08:50,040 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/a914cf73192e4f52a06cbac1f62d92d1, keycount=313518, bloomtype=NONE, size=32.5m, encoding=NONE
2012-11-09 00:08:50,041 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/571dc1e866184ca6aed3baa2d491940e, keycount=443292, bloomtype=NONE, size=47.3m, encoding=NONE
2012-11-09 00:08:50,076 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44with permission:rwxrwxrwx
2012-11-09 00:08:50,110 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:08:50,110 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44: CompoundBloomFilterWriter
2012-11-09 00:08:50,815 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:08:50,829 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.35 MB, total=550.21 MB, single=153.5 MB, multi=464.05 MB, memory=0 KB
2012-11-09 00:08:52,225 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 00:08:54,736 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44)
2012-11-09 00:08:54,744 INFO org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44 to hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/dbe372eda2fb4b76aa518af03f7bff44
2012-11-09 00:08:54,920 INFO org.apache.hadoop.hbase.regionserver.Store: Completed compaction of 3 file(s) in record_info of taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. into dbe372eda2fb4b76aa518af03f7bff44, size=129.0m; total size for store is 1.6g
2012-11-09 00:08:54,920 INFO org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: completed compaction: regionName=taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6., storeName=record_info, fileCount=3, fileSize=133.6m, priority=2, time=697165571949702; duration=4sec
2012-11-09 00:08:54,920 DEBUG org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: CompactSplitThread status: compaction_queue=(0:0), split_queue=0
2012-11-09 00:09:11,150 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:09:11,155 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=550.11 MB, single=148.66 MB, multi=468.77 MB, memory=0 KB
2012-11-09 00:09:28,449 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:09:28,460 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.09 MB, single=153.04 MB, multi=464.41 MB, memory=0 KB
2012-11-09 00:09:39,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=591.8 MB, free=141.58 MB, max=733.38 MB, blocks=9267, accesses=4794675147, hits=4790627441, hitRatio=99.91%, cachingAccesses=4794020702, cachingHits=4790603814, cachingHitsRatio=99.92%, evictions=2908, evicted=3407621, evictedPerRun=1171.8092041015625
2012-11-09 00:09:49,245 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:09:49,255 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.07 MB, single=149.17 MB, multi=468.28 MB, memory=0 KB
2012-11-09 00:10:12,186 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:10:12,191 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.18 MB, single=153.84 MB, multi=463.59 MB, memory=0 KB
2012-11-09 00:10:29,793 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.39 MB
2012-11-09 00:10:29,804 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.01 MB, single=159.33 MB, multi=458.03 MB, memory=0 KB
2012-11-09 00:10:49,676 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:10:49,683 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.43 MB, total=550.04 MB, single=144.88 MB, multi=472.51 MB, memory=0 KB
2012-11-09 00:11:09,801 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:11:09,813 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.14 MB, single=156.23 MB, multi=461.17 MB, memory=0 KB
2012-11-09 00:11:39,395 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.41 MB
2012-11-09 00:11:39,405 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.11 MB, single=147.77 MB, multi=469.61 MB, memory=0 KB
2012-11-09 00:12:02,520 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:12:02,529 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.1 MB, single=161.43 MB, multi=455.99 MB, memory=0 KB
2012-11-09 00:12:22,728 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:12:22,734 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.02 MB, single=142.63 MB, multi=474.76 MB, memory=0 KB
2012-11-09 00:12:50,996 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:12:51,007 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.43 MB, total=550.1 MB, single=156.5 MB, multi=460.88 MB, memory=0 KB
2012-11-09 00:13:09,661 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:13:09,673 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.31 MB, single=157.14 MB, multi=460.36 MB, memory=0 KB
2012-11-09 00:13:28,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:13:28,205 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.13 MB, single=148.29 MB, multi=469.07 MB, memory=0 KB
2012-11-09 00:13:46,681 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:13:46,692 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.2 MB, single=153.92 MB, multi=463.58 MB, memory=0 KB
2012-11-09 00:14:06,839 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:14:06,850 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=550.13 MB, single=148.52 MB, multi=468.94 MB, memory=0 KB
2012-11-09 00:14:26,835 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:14:26,846 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.04 MB, single=149.82 MB, multi=467.54 MB, memory=0 KB
2012-11-09 00:14:39,187 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=589.1 MB, free=144.28 MB, max=733.38 MB, blocks=9226, accesses=4817256651, hits=4813192706, hitRatio=99.91%, cachingAccesses=4816602220, cachingHits=4813169094, cachingHitsRatio=99.92%, evictions=2922, evicted=3423901, evictedPerRun=1171.7662353515625
2012-11-09 00:14:48,384 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:14:48,395 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.19 MB, single=150.69 MB, multi=466.84 MB, memory=0 KB
2012-11-09 00:15:06,866 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:15:06,871 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.09 MB, single=157.06 MB, multi=460.39 MB, memory=0 KB
2012-11-09 00:15:26,386 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:15:26,391 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.03 MB, single=142.63 MB, multi=474.77 MB, memory=0 KB
2012-11-09 00:15:46,206 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:15:46,212 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.06 MB, single=155.44 MB, multi=461.95 MB, memory=0 KB
2012-11-09 00:16:10,667 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:16:10,677 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.02 MB, single=151.52 MB, multi=465.86 MB, memory=0 KB
2012-11-09 00:16:43,503 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 00:16:43,503 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20. due to global heap pressure
2012-11-09 00:16:43,503 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20., current region memstore size 110.6m
2012-11-09 00:16:43,504 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20., commencing wait for mvcc, flushsize=115944576
2012-11-09 00:16:43,504 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 00:16:43,548 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 00:16:43,561 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91dwith permission:rwxrwxrwx
2012-11-09 00:16:43,622 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:16:43,622 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d: CompoundBloomFilterWriter
2012-11-09 00:16:44,524 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d)
2012-11-09 00:16:44,524 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=248865, memsize=87.5m, into tmp file hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d
2012-11-09 00:16:44,530 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d to hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/c66335aaedee4ed48ce08d2ed32ab91d
2012-11-09 00:16:44,580 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/c66335aaedee4ed48ce08d2ed32ab91d, entries=356415, sequenceid=248865, filesize=27.0m
2012-11-09 00:16:44,581 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~110.6m/115944576, currentsize=57.9k/59328 for region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20. in 1078ms, sequenceid=248865, compaction requested=false
2012-11-09 00:16:54,437 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.4 MB of total=623.43 MB
2012-11-09 00:16:54,447 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.45 MB, total=550.1 MB, single=153.49 MB, multi=463.92 MB, memory=0 KB
2012-11-09 00:17:15,232 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:17:15,243 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.18 MB, single=156.88 MB, multi=460.54 MB, memory=0 KB
2012-11-09 00:17:31,641 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:17:31,649 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.03 MB, single=155.66 MB, multi=461.73 MB, memory=0 KB
2012-11-09 00:17:55,192 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.43 MB
2012-11-09 00:17:55,197 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.22 MB, single=151.95 MB, multi=465.52 MB, memory=0 KB
2012-11-09 00:18:14,279 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:18:14,289 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.12 MB, single=146.87 MB, multi=470.56 MB, memory=0 KB
2012-11-09 00:18:31,892 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:18:31,903 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.11 MB, single=148.45 MB, multi=468.95 MB, memory=0 KB
2012-11-09 00:18:55,244 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:18:55,249 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.02 MB, single=149.58 MB, multi=467.82 MB, memory=0 KB
2012-11-09 00:19:14,647 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:19:14,652 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.12 MB, single=161.91 MB, multi=455.53 MB, memory=0 KB
2012-11-09 00:19:33,690 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:19:33,701 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.14 MB, single=146.9 MB, multi=470.48 MB, memory=0 KB
2012-11-09 00:19:39,193 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=573.16 MB, free=160.21 MB, max=733.38 MB, blocks=8974, accesses=4837602264, hits=4833522292, hitRatio=99.91%, cachingAccesses=4836947819, cachingHits=4833498665, cachingHitsRatio=99.92%, evictions=2936, evicted=3440181, evictedPerRun=1171.7237548828125
2012-11-09 00:19:54,689 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:19:54,700 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=550.1 MB, single=143.48 MB, multi=473.95 MB, memory=0 KB
2012-11-09 00:20:17,993 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:20:18,004 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.02 MB, single=152.53 MB, multi=464.87 MB, memory=0 KB
2012-11-09 00:20:56,068 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:20:56,079 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550 MB, single=153.66 MB, multi=463.69 MB, memory=0 KB
2012-11-09 00:21:20,070 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:21:20,081 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550 MB, single=150.06 MB, multi=467.29 MB, memory=0 KB
2012-11-09 00:21:39,357 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:21:39,368 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=549.97 MB, single=148.17 MB, multi=469.22 MB, memory=0 KB
2012-11-09 00:21:53,786 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.41 MB
2012-11-09 00:21:53,791 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.16 MB, single=153.64 MB, multi=463.75 MB, memory=0 KB
2012-11-09 00:22:09,045 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:22:09,050 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=550.19 MB, single=142.22 MB, multi=475.23 MB, memory=0 KB
2012-11-09 00:22:22,680 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:22:22,685 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.4 MB, total=549.99 MB, single=154.99 MB, multi=462.38 MB, memory=0 KB
2012-11-09 00:22:36,247 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.43 MB
2012-11-09 00:22:36,259 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.11 MB, single=151.06 MB, multi=466.42 MB, memory=0 KB
2012-11-09 00:22:47,994 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:22:48,006 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.11 MB, single=146.43 MB, multi=471.03 MB, memory=0 KB
2012-11-09 00:22:57,536 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.43 MB
2012-11-09 00:22:57,542 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.13 MB, single=138.95 MB, multi=478.52 MB, memory=0 KB
2012-11-09 00:23:07,835 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:23:07,845 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.06 MB, single=154.92 MB, multi=462.52 MB, memory=0 KB
2012-11-09 00:23:17,648 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.34 MB of total=623.37 MB
2012-11-09 00:23:17,659 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.38 MB, single=148.53 MB, multi=468.95 MB, memory=0 KB
2012-11-09 00:23:30,995 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:23:31,007 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.09 MB, single=142.48 MB, multi=474.91 MB, memory=0 KB
2012-11-09 00:23:46,183 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:23:46,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.02 MB, single=154.22 MB, multi=463.17 MB, memory=0 KB
2012-11-09 00:24:06,894 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:24:06,904 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.41 MB, total=549.98 MB, single=146.89 MB, multi=470.49 MB, memory=0 KB
2012-11-09 00:24:12,173 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 00:24:12,174 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. due to global heap pressure
2012-11-09 00:24:12,174 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38., current region memstore size 114.6m
2012-11-09 00:24:12,174 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38., commencing wait for mvcc, flushsize=120129544
2012-11-09 00:24:12,174 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 00:24:12,257 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684with permission:rwxrwxrwx
2012-11-09 00:24:12,304 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:24:12,304 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684: CompoundBloomFilterWriter
2012-11-09 00:24:12,432 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 00:24:13,179 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684)
2012-11-09 00:24:13,180 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=248944, memsize=80.3m, into tmp file hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684
2012-11-09 00:24:13,186 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684 to hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/a77e1905f21a4174b594d6f8d64cd684
2012-11-09 00:24:13,238 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/a77e1905f21a4174b594d6f8d64cd684, entries=326993, sequenceid=248944, filesize=24.8m
2012-11-09 00:24:13,239 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~114.6m/120129544, currentsize=37.1k/37960 for region taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. in 1065ms, sequenceid=248944, compaction requested=false
2012-11-09 00:24:26,978 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.42 MB
2012-11-09 00:24:26,989 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.29 MB, single=154.78 MB, multi=462.69 MB, memory=0 KB
2012-11-09 00:24:39,188 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=598.3 MB, free=135.08 MB, max=733.38 MB, blocks=9373, accesses=4865823971, hits=4861723825, hitRatio=99.91%, cachingAccesses=4865169535, cachingHits=4861700208, cachingHitsRatio=99.92%, evictions=2953, evicted=3459954, evictedPerRun=1171.6741943359375
2012-11-09 00:24:45,922 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:24:45,934 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.09 MB, single=147.02 MB, multi=470.38 MB, memory=0 KB
2012-11-09 00:25:04,214 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:25:04,225 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.21 MB, single=151.87 MB, multi=465.57 MB, memory=0 KB
2012-11-09 00:25:29,735 DEBUG org.apache.hadoop.hbase.regionserver.LogRoller: Hlog roll period 3600000ms elapsed
2012-11-09 00:25:29,788 DEBUG org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: using new createWriter -- HADOOP-6840
2012-11-09 00:25:29,789 DEBUG org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: Path=hdfs://master:54310/hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352391929736, syncFs=true, hflush=false, compression=false
2012-11-09 00:25:29,840 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Roll /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352388328705, entries=1867, filesize=4237942. for /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352391929736
2012-11-09 00:25:29,841 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Found 9 hlogs to remove out of total 22; oldest outstanding sequenceid is 181222 from region 5b2ca166dc37841bdecb51e4cb942914
2012-11-09 00:25:29,841 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352309103831 whose highest sequenceid is 177189 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352309103831
2012-11-09 00:25:29,922 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352312704819 whose highest sequenceid is 178134 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352312704819
2012-11-09 00:25:29,964 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352316305787 whose highest sequenceid is 178676 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352316305787
2012-11-09 00:25:30,006 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352319907042 whose highest sequenceid is 179352 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352319907042
2012-11-09 00:25:30,047 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352323508031 whose highest sequenceid is 179994 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352323508031
2012-11-09 00:25:30,113 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352327108994 whose highest sequenceid is 180535 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352327108994
2012-11-09 00:25:30,163 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352330709960 whose highest sequenceid is 180912 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352330709960
2012-11-09 00:25:30,198 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352334310944 whose highest sequenceid is 181137 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352334310944
2012-11-09 00:25:30,248 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352337912037 whose highest sequenceid is 181221 to /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352337912037
2012-11-09 00:25:37,177 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:25:37,183 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.16 MB, single=155.91 MB, multi=461.48 MB, memory=0 KB
2012-11-09 00:25:53,366 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:25:53,377 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.39 MB, total=550.06 MB, single=150.85 MB, multi=466.52 MB, memory=0 KB
2012-11-09 00:26:10,477 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.4 MB of total=623.43 MB
2012-11-09 00:26:10,484 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.05 MB, single=150.31 MB, multi=467.17 MB, memory=0 KB
2012-11-09 00:26:29,939 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:26:29,946 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=549.99 MB, single=152 MB, multi=465.41 MB, memory=0 KB
2012-11-09 00:26:49,262 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:26:49,271 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.1 MB, single=149.6 MB, multi=467.8 MB, memory=0 KB
2012-11-09 00:27:03,160 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:27:03,172 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.11 MB, single=152.89 MB, multi=464.64 MB, memory=0 KB
2012-11-09 00:27:27,736 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:27:27,747 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.43 MB, total=549.99 MB, single=143.24 MB, multi=474.17 MB, memory=0 KB
2012-11-09 00:27:47,871 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.4 MB of total=623.43 MB
2012-11-09 00:27:47,883 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.14 MB, single=148.23 MB, multi=469.25 MB, memory=0 KB
2012-11-09 00:28:14,121 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.36 MB of total=623.39 MB
2012-11-09 00:28:14,126 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.07 MB, single=150.18 MB, multi=467.25 MB, memory=0 KB
2012-11-09 00:28:38,799 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:28:38,811 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.06 MB, single=149.8 MB, multi=467.56 MB, memory=0 KB
2012-11-09 00:29:09,256 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:29:09,267 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.16 MB, single=148.07 MB, multi=469.39 MB, memory=0 KB
2012-11-09 00:29:25,085 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:29:25,097 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.04 MB, single=158.04 MB, multi=459.36 MB, memory=0 KB
2012-11-09 00:29:39,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=601.77 MB, free=131.6 MB, max=733.38 MB, blocks=9429, accesses=4887212513, hits=4883096023, hitRatio=99.91%, cachingAccesses=4886558080, cachingHits=4883072411, cachingHitsRatio=99.92%, evictions=2967, evicted=3476243, evictedPerRun=1171.6356201171875
2012-11-09 00:29:46,796 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:29:46,802 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.1 MB, single=143.15 MB, multi=474.25 MB, memory=0 KB
2012-11-09 00:30:03,306 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:30:03,317 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.22 MB, single=158.23 MB, multi=459.17 MB, memory=0 KB
2012-11-09 00:30:21,541 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:30:21,547 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.03 MB, single=147.05 MB, multi=470.33 MB, memory=0 KB
2012-11-09 00:30:40,859 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:30:40,864 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.03 MB, single=150.52 MB, multi=466.86 MB, memory=0 KB
2012-11-09 00:31:00,139 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:31:00,149 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.1 MB, single=152.87 MB, multi=464.53 MB, memory=0 KB
2012-11-09 00:31:09,372 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 00:31:09,372 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3. due to global heap pressure
2012-11-09 00:31:09,372 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3., current region memstore size 113.7m
2012-11-09 00:31:09,372 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3., commencing wait for mvcc, flushsize=119262296
2012-11-09 00:31:09,373 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 00:31:09,420 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541with permission:rwxrwxrwx
2012-11-09 00:31:09,461 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:31:09,461 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541: CompoundBloomFilterWriter
2012-11-09 00:31:10,911 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541)
2012-11-09 00:31:10,911 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=249027, memsize=113.7m, into tmp file hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541
2012-11-09 00:31:10,918 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541 to hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/item_info/bd3a73218c964f88b9a3c3878d9ba541
2012-11-09 00:31:10,965 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 00:31:10,977 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/item_info/bd3a73218c964f88b9a3c3878d9ba541, entries=422702, sequenceid=249027, filesize=41.9m
2012-11-09 00:31:10,980 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~113.7m/119262296, currentsize=21.9k/22456 for region taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3. in 1608ms, sequenceid=249027, compaction requested=false
2012-11-09 00:31:18,694 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:31:18,705 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.04 MB, single=149.53 MB, multi=467.93 MB, memory=0 KB
2012-11-09 00:31:36,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:31:36,205 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.1 MB, single=147.08 MB, multi=470.31 MB, memory=0 KB
2012-11-09 00:31:59,393 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 00:31:59,398 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.02 MB, single=147.63 MB, multi=469.75 MB, memory=0 KB
2012-11-09 00:32:20,362 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.4 MB of total=623.43 MB
2012-11-09 00:32:20,373 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.18 MB, single=154.1 MB, multi=463.44 MB, memory=0 KB
2012-11-09 00:32:41,480 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
2012-11-09 00:32:41,491 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=549.99 MB, single=151 MB, multi=466.41 MB, memory=0 KB
2012-11-09 00:33:03,782 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.39 MB of total=623.42 MB
....
2012-11-09 00:39:39,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=585.46 MB, free=147.91 MB, max=733.38 MB, blocks=9173, accesses=4935896422, hits=4931744129, hitRatio=99.91%, cachingAccesses=4935241977, cachingHits=4931720502, cachingHitsRatio=99.92%, evictions=2998, evicted=3512303, evictedPerRun=1171.5487060546875
2012-11-09 00:39:49,782 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:39:49,793 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.19 MB, single=153.7 MB, multi=463.79 MB, memory=0 KB
2012-11-09 00:40:09,319 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:40:09,329 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.07 MB, single=151.11 MB, multi=466.25 MB, memory=0 KB
2012-11-09 00:40:26,066 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
2012-11-09 00:40:26,077 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.37 MB, total=550.38 MB, single=156.6 MB, multi=460.83 MB, memory=0 KB
2012-11-09 00:40:33,396 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 00:40:33,396 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f. due to global heap pressure
2012-11-09 00:40:33,396 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f., current region memstore size 137.5m
2012-11-09 00:40:33,396 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f., commencing wait for mvcc, flushsize=144184360
2012-11-09 00:40:33,396 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 00:40:33,444 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 00:40:33,481 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94with permission:rwxrwxrwx
2012-11-09 00:40:33,516 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:40:33,516 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94: CompoundBloomFilterWriter
2012-11-09 00:40:34,548 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94)
2012-11-09 00:40:34,548 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=249054, memsize=89.6m, into tmp file hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94
2012-11-09 00:40:34,553 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94 to hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/f94b49bad2884cf38b19bf173e314e94
2012-11-09 00:40:34,612 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/f94b49bad2884cf38b19bf173e314e94, entries=364696, sequenceid=249054, filesize=27.7m
2012-11-09 00:40:34,613 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~137.5m/144184360, currentsize=65.2k/66816 for region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f. in 1217ms, sequenceid=249054, compaction requested=false
2012-11-09 00:40:46,468 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.35 MB of total=623.38 MB
.....
2012-11-09 00:51:20,609 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.05 MB, single=151.84 MB, multi=465.54 MB, memory=0 KB
2012-11-09 00:51:59,887 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:51:59,898 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.1 MB, single=150.96 MB, multi=466.43 MB, memory=0 KB
2012-11-09 00:52:47,163 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:52:47,174 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.1 MB, single=153.11 MB, multi=464.35 MB, memory=0 KB
2012-11-09 00:52:53,378 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 00:52:53,379 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20. due to global heap pressure
2012-11-09 00:52:53,379 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20., current region memstore size 126.8m
2012-11-09 00:52:53,379 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20., commencing wait for mvcc, flushsize=132907720
2012-11-09 00:52:53,379 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 00:52:53,385 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 00:52:53,452 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217with permission:rwxrwxrwx
2012-11-09 00:52:53,486 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 00:52:53,486 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217: CompoundBloomFilterWriter
2012-11-09 00:52:54,378 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217)
2012-11-09 00:52:54,378 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=249097, memsize=89.1m, into tmp file hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217
2012-11-09 00:52:54,385 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217 to hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/f3cb0947202d40909ac731edfcaac217
2012-11-09 00:52:54,435 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/f3cb0947202d40909ac731edfcaac217, entries=363251, sequenceid=249097, filesize=27.5m
2012-11-09 00:52:54,438 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~126.8m/132907720, currentsize=74.0k/75760 for region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20. in 1058ms, sequenceid=249097, compaction requested=false
2012-11-09 00:53:28,007 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:53:28,018 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.44 MB, total=550.03 MB, single=154.49 MB, multi=462.96 MB, memory=0 KB
2012-11-09 00:54:08,504 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.38 MB of total=623.41 MB
2012-11-09 00:54:08,515 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.38 MB, total=550.09 MB, single=148.56 MB, multi=468.89 MB, memory=0 KB
2012-11-09 00:54:39,180 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=596.52 MB, free=136.85 MB, max=733.38 MB, blocks=9346, accesses=4984056793, hits=4979865932, hitRatio=99.91%, cachingAccesses=4983402348, cachingHits=4979842305, cachingHitsRatio=99.92%, evictions=3031, evicted=3550697, evictedPerRun=1171.4605712890625
2012-11-09 00:54:56,875 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
......
2012-11-09 01:02:37,188 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.43 MB, total=549.98 MB, single=151.59 MB, multi=465.81 MB, memory=0 KB
2012-11-09 01:03:12,945 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 01:03:12,946 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. due to global heap pressure
2012-11-09 01:03:12,946 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38., current region memstore size 103.3m
2012-11-09 01:03:12,946 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38., commencing wait for mvcc, flushsize=108273864
2012-11-09 01:03:12,946 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 01:03:13,014 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2with permission:rwxrwxrwx
2012-11-09 01:03:13,047 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 01:03:13,047 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2: CompoundBloomFilterWriter
2012-11-09 01:03:13,155 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 01:03:13,754 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2)
2012-11-09 01:03:13,754 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=249293, memsize=67.4m, into tmp file hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2
2012-11-09 01:03:13,760 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2 to hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/54db1c93a33b46ba9e0c06c06a894ff2
2012-11-09 01:03:13,810 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/54db1c93a33b46ba9e0c06c06a894ff2, entries=274258, sequenceid=249293, filesize=20.8m
2012-11-09 01:03:13,812 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~103.3m/108273864, currentsize=17.7k/18144 for region taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. in 866ms, sequenceid=249293, compaction requested=true
2012-11-09 01:03:13,812 DEBUG org.apache.hadoop.hbase.regionserver.Store: 1469bafdb43e6d2eea476b9619921d38 - item_info: Initiating minorcompaction
2012-11-09 01:03:13,813 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on item_info in region taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
2012-11-09 01:03:13,813 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small Compaction requested: regionName=taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38., storeName=item_info, fileCount=4, fileSize=317.9m (251.4m, 20.9m, 24.8m, 20.8m), priority=3, time=700429344373702; Because: regionserver60020.cacheFlusher; compaction_queue=(0:0), split_queue=0
2012-11-09 01:03:13,813 INFO org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 4 file(s) in item_info of taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. into tmpdir=hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp, seqid=249293, totalSize=317.9m
2012-11-09 01:03:13,813 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/70ffe3f371e84c94b021cf087f6e9df6, keycount=3422770, bloomtype=NONE, size=251.4m, encoding=NONE
2012-11-09 01:03:13,813 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/6974289ad68e40ce81fd73cef25b20ab, keycount=274512, bloomtype=NONE, size=20.9m, encoding=NONE
2012-11-09 01:03:13,813 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/a77e1905f21a4174b594d6f8d64cd684, keycount=326993, bloomtype=NONE, size=24.8m, encoding=NONE
2012-11-09 01:03:13,814 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compacting hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/54db1c93a33b46ba9e0c06c06a894ff2, keycount=274258, bloomtype=NONE, size=20.8m, encoding=NONE
2012-11-09 01:03:13,829 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2cwith permission:rwxrwxrwx
2012-11-09 01:03:13,910 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 01:03:13,911 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2c: CompoundBloomFilterWriter
2012-11-09 01:03:19,620 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 73.37 MB of total=623.4 MB
2012-11-09 01:03:19,631 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=73.42 MB, total=550.24 MB, single=125.78 MB, multi=491.61 MB, memory=0 KB
2012-11-09 01:03:22,465 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2c)
2012-11-09 01:03:22,471 INFO org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2c to hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/585b2b294a5245df88230e666c624b2c
2012-11-09 01:03:22,722 INFO org.apache.hadoop.hbase.regionserver.Store: Completed major compaction of 4 file(s) in item_info of taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38. into 585b2b294a5245df88230e666c624b2c, size=255.5m; total size for store is 255.5m
2012-11-09 01:03:22,722 INFO org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: completed compaction: regionName=taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38., storeName=item_info, fileCount=4, fileSize=317.9m, priority=3, time=700429344373702; duration=8sec
2012-11-09 01:03:22,722 DEBUG org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: CompactSplitThread status: compaction_queue=(0:0), split_queue=0
.................
2012-11-09 01:24:39,187 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=483.42 MB, free=249.95 MB, max=733.38 MB, blocks=7555, accesses=5003183182, hits=4998959719, hitRatio=99.91%, cachingAccesses=5002509512, cachingHits=4998933235, cachingHitsRatio=99.92%, evictions=3044, evicted=3568722, evictedPerRun=1172.379150390625
2012-11-09 01:25:31,199 DEBUG org.apache.hadoop.hbase.regionserver.LogRoller: Hlog roll period 3600000ms elapsed
2012-11-09 01:25:31,253 DEBUG org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: using new createWriter -- HADOOP-6840
2012-11-09 01:25:31,254 DEBUG org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: Path=hdfs://master:54310/hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352395531200, syncFs=true, hflush=false, compression=false
2012-11-09 01:25:31,302 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Roll /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352391929736, entries=632, filesize=415028. for /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352395531200
2012-11-09 01:29:39,193 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=488.76 MB, free=244.62 MB, max=733.38 MB, blocks=7639, accesses=5003187645, hits=4998964098, hitRatio=99.91%, cachingAccesses=5002513975, cachingHits=4998937614, cachingHitsRatio=99.92%, evictions=3044, evicted=3568722, evictedPerRun=1172.379150390625
2012-11-09 01:34:39,187 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=494.67 MB, free=238.7 MB, max=733.38 MB, blocks=7732, accesses=5003192070, hits=4998968430, hitRatio=99.91%, cachingAccesses=5002518400, cachingHits=4998941946, cachingHitsRatio=99.92%, evictions=3044, evicted=3568722, evictedPerRun=1172.379150390625
2012-11-09 01:37:32,605 DEBUG org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up because memory above low water=1.0g
2012-11-09 01:37:32,605 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f. due to global heap pressure
2012-11-09 01:37:32,606 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f., current region memstore size 157.8m
2012-11-09 01:37:32,606 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f., commencing wait for mvcc, flushsize=165513280
2012-11-09 01:37:32,606 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencing flushing stores
2012-11-09 01:37:32,683 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating file:hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8with permission:rwxrwxrwx
2012-11-09 01:37:32,720 INFO org.apache.hadoop.hbase.regionserver.HRegion: writing data to region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f. with WAL disabled. Data may be lost in the event of a crash.
2012-11-09 01:37:32,731 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-11-09 01:37:32,731 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter type for hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8: CompoundBloomFilterWriter
2012-11-09 01:37:33,707 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO DeleteFamily was added to HFile (hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8)
2012-11-09 01:37:33,708 INFO org.apache.hadoop.hbase.regionserver.Store: Flushed , sequenceid=249835, memsize=101.6m, into tmp file hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8
2012-11-09 01:37:33,714 DEBUG org.apache.hadoop.hbase.regionserver.Store: Renaming flushed file at hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8 to hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/50dc3ad378ef45068d8af6e53e6951a8
2012-11-09 01:37:33,763 INFO org.apache.hadoop.hbase.regionserver.Store: Added hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/50dc3ad378ef45068d8af6e53e6951a8, entries=413606, sequenceid=249835, filesize=31.4m
2012-11-09 01:37:33,765 INFO org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~157.8m/165513280, currentsize=47.6k/48696 for region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f. in 1159ms, sequenceid=249835, compaction requested=false
2012-11-09 01:39:39,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=499.61 MB, free=233.76 MB, max=733.38 MB, blocks=7810, accesses=5003196649, hits=4998972931, hitRatio=99.91%, cachingAccesses=5002522979, cachingHits=4998946447, cachingHitsRatio=99.92%, evictions=3044, evicted=3568722, evictedPerRun=1172.379150390625
2012-11-09 01:43:12,097 INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard from server in 40000ms for sessionid 0x13ada9d738a0001, closing socket connection and attempting reconnect
2012-11-09 01:43:12,204 INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard from server in 40000ms for sessionid 0x13ada9d738a000b, closing socket connection and attempting reconnect
2012-11-09 01:43:12,897 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server /192.168.2.180:2222
2012-11-09 01:43:12,897 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:13,042 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave180/192.168.2.180:2222
2012-11-09 01:43:13,042 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:21,468 INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard from server in 9162ms for sessionid 0x13ada9d738a000b, closing socket connection and attempting reconnect
2012-11-09 01:43:21,613 INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard from server in 9414ms for sessionid 0x13ada9d738a0001, closing socket connection and attempting reconnect
2012-11-09 01:43:22,283 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave182/192.168.2.182:2222
2012-11-09 01:43:22,283 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:22,572 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server /192.168.2.182:2222
2012-11-09 01:43:22,572 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:22,573 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a000b for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:23,301 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave184/192.168.2.184:2222
2012-11-09 01:43:23,301 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:23,302 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a000b for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:24,142 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave185/192.168.2.185:2222
2012-11-09 01:43:24,142 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:24,143 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a000b for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:25,146 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave183/192.168.2.183:2222
2012-11-09 01:43:25,146 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:25,147 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a000b for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:25,283 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:25,613 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave181/192.168.2.181:2222
2012-11-09 01:43:25,613 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:25,614 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:25,782 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave181/192.168.2.181:2222
2012-11-09 01:43:25,782 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:25,783 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a000b for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:26,269 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave185/192.168.2.185:2222
2012-11-09 01:43:26,269 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:26,270 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:26,529 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave184/192.168.2.184:2222
2012-11-09 01:43:26,529 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:26,529 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:27,072 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master/192.168.2.167:2222
2012-11-09 01:43:27,072 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:27,073 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to master/192.168.2.167:2222, initiating session
2012-11-09 01:43:27,093 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server master/192.168.2.167:2222, sessionid = 0x13ada9d738a000b, negotiated timeout = 60000
2012-11-09 01:43:27,584 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave183/192.168.2.183:2222
2012-11-09 01:43:27,584 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:36,155 INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard from server in 9523ms for sessionid 0x13ada9d738a0001, closing socket connection and attempting reconnect
2012-11-09 01:43:38,109 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master/192.168.2.167:2222
2012-11-09 01:43:38,109 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:46,680 INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard from server in 10423ms for sessionid 0x13ada9d738a0001, closing socket connection and attempting reconnect
2012-11-09 01:43:47,537 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave180/192.168.2.180:2222
2012-11-09 01:43:47,537 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:49,126 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: master/192.168.2.167:54310. Already tried 0 time(s).
2012-11-09 01:43:50,537 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:50,979 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave182/192.168.2.182:2222
2012-11-09 01:43:50,979 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:50,980 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:51,234 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave181/192.168.2.181:2222
2012-11-09 01:43:51,234 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:51,235 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:51,452 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave185/192.168.2.185:2222
2012-11-09 01:43:51,452 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:51,452 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:52,221 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave184/192.168.2.184:2222
2012-11-09 01:43:52,221 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:52,222 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:52,352 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave183/192.168.2.183:2222
2012-11-09 01:43:52,352 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:52,352 WARN org.apache.zookeeper.ClientCnxn: Session 0x13ada9d738a0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2012-11-09 01:43:53,358 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server slave181,60020,1352287478815: Unhandled exception: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing slave181,60020,1352287478815 as dead server
org.apache.hadoop.hbase.YouAreDeadException: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing slave181,60020,1352287478815 as dead server
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:852)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:723)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing slave181,60020,1352287478815 as dead server
at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:242)
at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:160)
at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:935)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1400)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1021)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
at $Proxy8.regionServerReport(Unknown Source)
at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:849)
... 2 more
2012-11-09 01:43:53,374 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2012-11-09 01:43:53,378 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=0, numberOfOnlineRegions=34, numberOfStores=37, numberOfStorefiles=52, storefileIndexSizeMB=1, rootIndexSizeKB=1084, totalStaticIndexSizeKB=29868, totalStaticBloomSizeKB=0, memstoreSizeMB=919, mbInMemoryWithoutWAL=286, numberOfPutsWithoutWAL=3532813, readRequestsCount=1299409247, writeRequestsCount=15476790, compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1575, maxHeapMB=2933, blockCacheSizeMB=502.99, blockCacheFreeMB=230.39, blockCacheCount=7863, blockCacheHitCount=4998976142, blockCacheMissCount=4223771, blockCacheEvictedCount=3568722, blockCacheHitRatio=99%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=97, slowHLogAppendCount=0, fsReadLatencyHistogramMean=1488798.95, fsReadLatencyHistogramCount=2956662, fsReadLatencyHistogramMedian=239500, fsReadLatencyHistogram75th=287750, fsReadLatencyHistogram95th=654900, fsReadLatencyHistogram99th=1532570, fsReadLatencyHistogram999th=26074061, fsPreadLatencyHistogramMean=3522484.19, fsPreadLatencyHistogramCount=1264015, fsPreadLatencyHistogramMedian=1426500, fsPreadLatencyHistogram75th=1778250, fsPreadLatencyHistogram95th=15080950, fsPreadLatencyHistogram99th=25166530, fsPreadLatencyHistogram999th=114129728, fsWriteLatencyHistogramMean=1253190.67, fsWriteLatencyHistogramCount=747208, fsWriteLatencyHistogramMedian=409500, fsWriteLatencyHistogram75th=653500, fsWriteLatencyHistogram95th=2717950, fsWriteLatencyHistogram99th=4138890, fsWriteLatencyHistogram999th=41532109
2012-11-09 01:43:53,398 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled exception: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing slave181,60020,1352287478815 as dead server
2012-11-09 01:43:53,398 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
....
2012-11-09 01:43:53,403 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 60020
2012-11-09 01:43:53,405 INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker: Sending interrupt to stop the worker thread
2012-11-09 01:43:53,405 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
2012-11-09 01:43:53,405 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2012-11-09 01:43:53,415 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 30 on 60020: exiting
...
2012-11-09 01:43:53,405 INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker: SplitLogWorker interrupted while waiting for task, exiting: java.lang.InterruptedException
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 38 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 40 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 39 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker: SplitLogWorker slave181,60020,1352287478815 exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 41 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 37 on 60020: exiting
2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 34 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 47 on 60020: exiting
2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 33 on 60020: exiting
2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 18 on 60020: exiting
2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 32 on 60020: exiting
2012-11-09 01:43:53,415 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 49 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 48 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 46 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 45 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 43 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 36 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 44 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 31 on 60020: exiting
2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 42 on 60020: exiting
2012-11-09 01:43:53,539 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: regionserver60020.cacheFlusher exiting
2012-11-09 01:43:53,539 INFO org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
2012-11-09 01:43:53,540 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free -48043.02 KB of total=503.11 MB
2012-11-09 01:43:53,540 INFO org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionChecker: regionserver60020.compactionChecker exiting
2012-11-09 01:43:53,540 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
2012-11-09 01:43:53,540 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.: disabling compactions & flushes
2012-11-09 01:43:53,541 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
2012-11-09 01:43:53,541 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
2012-11-09 01:43:53,541 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.: disabling compactions & flushes
2012-11-09 01:43:53,541 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
2012-11-09 01:43:53,542 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: aborting server slave181,60020,1352287478815
2012-11-09 01:43:53,542 DEBUG org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@66447828
2012-11-09 01:43:53,543 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x13ada9d738a000b
2012-11-09 01:43:53,543 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97 \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
2012-11-09 01:43:53,543 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97 \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.: disabling compactions & flushes
2012-11-09 01:43:53,543 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97 \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
2012-11-09 01:43:53,544 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed record_info
2012-11-09 01:43:53,544 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
2012-11-09 01:43:53,544 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
2012-11-09 01:43:53,544 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
2012-11-09 01:43:53,544 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed search_info
2012-11-09 01:43:53,544 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.: disabling compactions & flushes
2012-11-09 01:43:53,545 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
2012-11-09 01:43:53,545 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
2012-11-09 01:43:53,545 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
2012-11-09 01:43:53,545 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
2012-11-09 01:43:53,545 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.: disabling compactions & flushes
2012-11-09 01:43:53,545 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
2012-11-09 01:43:53,546 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed search_info
2012-11-09 01:43:53,546 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97 \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
2012-11-09 01:43:53,546 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97 \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
2012-11-09 01:43:53,546 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
2012-11-09 01:43:53,547 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.: disabling compactions & flushes
2012-11-09 01:43:53,547 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
2012-11-09 01:43:53,547 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed redirect_info
2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed redirect_info
2012-11-09 01:43:53,548 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
2012-11-09 01:43:53,548 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.: disabling compactions & flushes
2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
2012-11-09 01:43:53,549 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.: disabling compactions & flushes
2012-11-09 01:43:53,549 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
2012-11-09 01:43:53,549 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed record_info
2012-11-09 01:43:53,550 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
2012-11-09 01:43:53,550 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
2012-11-09 01:43:53,550 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
2012-11-09 01:43:53,550 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.: disabling compactions & flushes
2012-11-09 01:43:53,550 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
2012-11-09 01:43:53,555 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed record_info
2012-11-09 01:43:53,555 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
2012-11-09 01:43:53,555 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
2012-11-09 01:43:53,555 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
2012-11-09 01:43:53,555 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.: disabling compactions & flushes
2012-11-09 01:43:53,555 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
2012-11-09 01:43:53,557 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed redirect_info
2012-11-09 01:43:53,557 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed redirect_info
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.: disabling compactions & flushes
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
2012-11-09 01:43:53,558 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.: disabling compactions & flushes
2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
2012-11-09 01:43:53,559 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed record_info
2012-11-09 01:43:53,559 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
2012-11-09 01:43:53,560 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
2012-11-09 01:43:53,560 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
2012-11-09 01:43:53,560 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.: disabling compactions & flushes
2012-11-09 01:43:53,560 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
2012-11-09 01:43:53,566 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed item_info
2012-11-09 01:43:53,566 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
2012-11-09 01:43:53,566 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
2012-11-09 01:43:53,566 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
2012-11-09 01:43:53,566 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.: disabling compactions & flushes
2012-11-09 01:43:53,566 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
2012-11-09 01:43:53,568 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed record_info
2012-11-09 01:43:53,569 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
2012-11-09 01:43:53,569 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
2012-11-09 01:43:53,569 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
2012-11-09 01:43:53,569 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.: disabling compactions & flushes
2012-11-09 01:43:53,569 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
2012-11-09 01:43:53,570 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed doc_info
2012-11-09 01:43:53,570 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed doc_pic
2012-11-09 01:43:53,573 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed item_info
2012-11-09 01:43:53,574 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
2012-11-09 01:43:53,574 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
2012-11-09 01:43:53,574 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
2012-11-09 01:43:53,574 INFO org.apache.zookeeper.ZooKeeper: Session: 0x13ada9d738a000b closed
2012-11-09 01:43:53,574 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2012-11-09 01:43:53,574 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.: disabling compactions & flushes
2012-11-09 01:43:53,574 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on 21 regions to close
2012-11-09 01:43:53,574 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
2012-11-09 01:43:53,575 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed doc_stat
2012-11-09 01:43:53,575 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed reply_data
2012-11-09 01:43:53,576 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
2012-11-09 01:43:53,576 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
2012-11-09 01:43:53,576 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
2012-11-09 01:43:53,576 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.: disabling compactions & flushes
2012-11-09 01:43:53,576 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
2012-11-09 01:43:53,578 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed search_info
2012-11-09 01:43:53,578 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
2012-11-09 01:43:53,579 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
2012-11-09 01:43:53,579 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
2012-11-09 01:43:53,579 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.: disabling compactions & flushes
2012-11-09 01:43:53,579 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
2012-11-09 01:43:53,590 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed item_info
2012-11-09 01:43:53,590 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
2012-11-09 01:43:53,590 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
2012-11-09 01:43:53,590 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
2012-11-09 01:43:53,591 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.: disabling compactions & flushes
2012-11-09 01:43:53,591 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
2012-11-09 01:43:53,593 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed img_info
2012-11-09 01:43:53,593 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
2012-11-09 01:43:53,593 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
2012-11-09 01:43:53,593 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
2012-11-09 01:43:53,593 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.: disabling compactions & flushes
2012-11-09 01:43:53,593 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
2012-11-09 01:43:53,597 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed item_info
2012-11-09 01:43:53,597 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
2012-11-09 01:43:53,597 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
2012-11-09 01:43:53,597 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
2012-11-09 01:43:53,597 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.: disabling compactions & flushes
2012-11-09 01:43:53,597 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
2012-11-09 01:43:53,598 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed info
2012-11-09 01:43:53,598 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
2012-11-09 01:43:53,598 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
2012-11-09 01:43:53,598 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
2012-11-09 01:43:53,598 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.: disabling compactions & flushes
2012-11-09 01:43:53,598 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
2012-11-09 01:43:53,599 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed img_info
2012-11-09 01:43:53,599 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
2012-11-09 01:43:53,599 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
2012-11-09 01:43:53,600 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
2012-11-09 01:43:53,600 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.: disabling compactions & flushes
2012-11-09 01:43:53,600 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
2012-11-09 01:43:53,603 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed info
2012-11-09 01:43:53,603 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
2012-11-09 01:43:53,603 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
2012-11-09 01:43:53,603 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
2012-11-09 01:43:53,603 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.: disabling compactions & flushes
2012-11-09 01:43:53,603 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
2012-11-09 01:43:53,605 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed item_info
2012-11-09 01:43:53,606 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed record_info
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_search_keyword_table,20121103|\xE7\x94\xB7\xE5\xA3\xAB \xE8\x83\x8C\xE5\xBF\x83 \xE5\xAE\xBD\xE6\x9D\xBE|1_24,1351960783892.5902777356e5a6994b0e267991957df0.
2012-11-09 01:43:53,606 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_search_keyword_table,20121103|\xE7\x94\xB7\xE5\xA3\xAB \xE8\x83\x8C\xE5\xBF\x83 \xE5\xAE\xBD\xE6\x9D\xBE|1_24,1351960783892.5902777356e5a6994b0e267991957df0.: disabling compactions & flushes
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_search_keyword_table,20121103|\xE7\x94\xB7\xE5\xA3\xAB \xE8\x83\x8C\xE5\xBF\x83 \xE5\xAE\xBD\xE6\x9D\xBE|1_24,1351960783892.5902777356e5a6994b0e267991957df0.
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_table,20121103|\xE9\x99\xB6\xE7\x93\xB7\xE6\xB0\xB4\xE6\xA7\xBD\xE6\xB4\x97\xE8\xA1\xA3\xE7\x9B\x86|110,1352020981378.50241b8ed071b624f785064cac238367.
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_table,20121103|\xE9\x99\xB6\xE7\x93\xB7\xE6\xB0\xB4\xE6\xA7\xBD\xE6\xB4\x97\xE8\xA1\xA3\xE7\x9B\x86|110,1352020981378.50241b8ed071b624f785064cac238367.: disabling compactions & flushes
2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_table,20121103|\xE9\x99\xB6\xE7\x93\xB7\xE6\xB0\xB4\xE6\xA7\xBD\xE6\xB4\x97\xE8\xA1\xA3\xE7\x9B\x86|110,1352020981378.50241b8ed071b624f785064cac238367.
2012-11-09 01:43:53,607 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed record_info
2012-11-09 01:43:53,607 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
2012-11-09 01:43:53,607 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
2012-11-09 01:43:53,607 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
2012-11-09 01:43:53,608 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.: disabling compactions & flushes
2012-11-09 01:43:53,608 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
2012-11-09 01:43:53,609 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed top_info
2012-11-09 01:43:53,610 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
2012-11-09 01:43:53,610 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
2012-11-09 01:43:53,610 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
2012-11-09 01:43:53,610 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.: disabling compactions & flushes
2012-11-09 01:43:53,610 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
2012-11-09 01:43:53,613 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed item_info
2012-11-09 01:43:53,613 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
2012-11-09 01:43:53,613 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
2012-11-09 01:43:53,614 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
2012-11-09 01:43:53,614 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.: disabling compactions & flushes
2012-11-09 01:43:53,614 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
2012-11-09 01:43:53,616 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed redirect_info
2012-11-09 01:43:53,616 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
2012-11-09 01:43:53,616 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
2012-11-09 01:43:53,616 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing close of taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
2012-11-09 01:43:53,616 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.: disabling compactions & flushes
2012-11-09 01:43:53,616 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97 \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
2012-11-09 01:43:53,619 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed redirect_info
....
2012-11-09 01:43:53,625 DEBUG org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed region taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3.
2012-11-09 01:43:53,696 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master/192.168.2.167:2222
2012-11-09 01:43:53,696 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-11-09 01:43:53,696 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to master/192.168.2.167:2222, initiating session
2012-11-09 01:43:53,697 INFO org.apache.zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x13ada9d738a0001 has expired, closing socket connection
2012-11-09 01:43:53,697 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server slave181,60020,1352287478815: regionserver:60020-0x13ada9d738a0001 regionserver:60020-0x13ada9d738a0001 received expired from ZooKeeper, aborting
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:389)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:286)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
2012-11-09 01:43:53,700 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2012-11-09 01:43:53,702 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=0, numberOfOnlineRegions=34, numberOfStores=37, numberOfStorefiles=52, storefileIndexSizeMB=1, rootIndexSizeKB=1084, totalStaticIndexSizeKB=29868, totalStaticBloomSizeKB=0, memstoreSizeMB=919, mbInMemoryWithoutWAL=286, numberOfPutsWithoutWAL=3532813, readRequestsCount=1299409247, writeRequestsCount=15476790, compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1601, maxHeapMB=2933, blockCacheSizeMB=502.99, blockCacheFreeMB=230.39, blockCacheCount=7863, blockCacheHitCount=4998976142, blockCacheMissCount=4223771, blockCacheEvictedCount=3568722, blockCacheHitRatio=99%, blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=97, slowHLogAppendCount=0, fsReadLatencyHistogramMean=1488798.95, fsReadLatencyHistogramCount=2956662, fsReadLatencyHistogramMedian=239500, fsReadLatencyHistogram75th=287750, fsReadLatencyHistogram95th=654900, fsReadLatencyHistogram99th=1532570, fsReadLatencyHistogram999th=26074061, fsPreadLatencyHistogramMean=3522484.19, fsPreadLatencyHistogramCount=1264015, fsPreadLatencyHistogramMedian=1426500, fsPreadLatencyHistogram75th=1778250, fsPreadLatencyHistogram95th=15080950, fsPreadLatencyHistogram99th=25166530, fsPreadLatencyHistogram999th=114129728, fsWriteLatencyHistogramMean=1253190.67, fsWriteLatencyHistogramCount=747208, fsWriteLatencyHistogramMedian=409500, fsWriteLatencyHistogram75th=653500, fsWriteLatencyHistogram95th=2717950, fsWriteLatencyHistogram99th=4138890, fsWriteLatencyHistogram999th=41532109
2012-11-09 01:43:53,708 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: regionserver:60020-0x13ada9d738a0001 regionserver:60020-0x13ada9d738a0001 received expired from ZooKeeper, aborting
2012-11-09 01:43:53,708 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2012-11-09 01:43:53,776 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server slave181,60020,1352287478815; all regions closed.
2012-11-09 01:43:53,778 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: regionserver60020.logSyncer exiting
2012-11-09 01:43:53,781 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: closing hlog writer in hdfs://master:54310/hbase/.logs/slave181,60020,1352287478815
2012-11-09 01:43:53,839 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Close and delete failed
org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352395531200 File does not exist. [Lease. Holder: DFSClient_hb_rs_slave181,60020,1352287478815, pendingcreates: 1]
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1631)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1622)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:1677)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:1665)
at org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:718)
at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
at org.apache.hadoop.hbase.regionserver.HRegionServer.closeWAL(HRegionServer.java:945)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:800)
at java.lang.Thread.run(Thread.java:722)
2012-11-09 01:43:53,840 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver60020 closing leases
2012-11-09 01:43:53,840 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver60020 closed leases
2012-11-09 01:43:53,840 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting for Split Thread to finish...
2012-11-09 01:43:53,840 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting for Large Compaction Thread to finish...
2012-11-09 01:43:53,840 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting for Small Compaction Thread to finish...
2012-11-09 01:43:53,850 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
2012-11-09 01:43:53,850 INFO org.apache.hadoop.hbase.util.RetryCounter: Sleeping 2000ms before retry #1...
2012-11-09 01:43:55,478 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver60020.leaseChecker closing leases
2012-11-09 01:43:55,481 INFO org.apache.hadoop.hbase.regionserver.Leases: regionserver60020.leaseChecker closed leases
2012-11-09 01:43:55,852 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
2012-11-09 01:43:55,852 INFO org.apache.hadoop.hbase.util.RetryCounter: Sleeping 4000ms before retry #2...
2012-11-09 01:43:59,854 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
2012-11-09 01:43:59,854 INFO org.apache.hadoop.hbase.util.RetryCounter: Sleeping 8000ms before retry #3...
2012-11-09 01:44:07,856 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
2012-11-09 01:44:07,856 ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper delete failed after 3 retries
2012-11-09 01:44:07,856 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: Failed deleting my ephemeral node
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:868)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:127)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:974)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:963)
at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1037)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:815)
at java.lang.Thread.run(Thread.java:722)
2012-11-09 01:44:07,857 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server slave181,60020,1352287478815; zookeeper connection closed.
2012-11-09 01:44:07,857 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver60020 exiting
2012-11-09 01:44:07,875 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-5,5,main]
2012-11-09 01:44:07,875 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Shutdown hook
2012-11-09 01:44:07,875 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs shutdown hook thread.
2012-11-09 01:44:07,876 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook finished
LOG END--------------------------------------------------------------------------------------------------------
--------------
兰锋成
Tel: 0592-5900539/Fax: 0592-5900395
Mobel: 13599927214
Email: lanfengcheng@xmtongrong.com
Website: www.xmtongrong.com
厦门瑞景商业广场 1#6 层 1-601 单元
Room601, building No.1 Ruijing Shopping Mall, Siming District, Amoy
Re: hbase slave region server is terminated unexpectedly
Posted by ramkrishna vasudevan <ra...@gmail.com>.
What is your configured heap size? Check your GC logs. Most probabaly a
full GC has happened.
Regards
Ram
On Fri, Nov 9, 2012 at 8:07 AM, lanfengcheng@xmtr <
lanfengcheng@xmtongrong.com> wrote:
> HI,
> Recently our HBASE cluster meet a situation that one of our
> HRegionServer nodes (random one) stop unexpectedly .
> our HBASE servers are in high concurrent write data and read and write
> data.
> At first we thought it may be caused due to configuration problem, we
> therefore tried the optimization of HBASE parameter .
> But it still happended after the optimization of the parameters, the
> process termination of HRegionServer.
> Please help us and give some suggestions or solutions, thank you.
>
> Environment:
> My HBase culster contains 7 computers. One is the master and zookeeper
> server, the other 6 are the region servers.
> The operatiing system is Centos 5.6 with kenerl 2.6.18-238.el5.
> The jdk version is 1.7.0_03
> HBASE VERSION:0.94.2
> HADOOP VERSION:Hadoop 1.0.4
>
> PARTS OF HBASE-SITE.XML CONFIGURATIONS:
> ----------------------------------------------------------------------
> <property>
> <name>hbase.regionserver.lease.period</name>
> <value>180000</value>
> </property>
> <property>
> <name>hbase.hregion.max.filesize</name>
> <value>2048000000</value>
> </property>
> <property>
> <name>hbase.hregion.memstore.flush.size</name>
> <value>512000000</value>
> </property>
> <property>
> <name>zookeeper.session.timeout</name>
> <value>60000</value>
> </property>
> <property>
> <name>hbase.regionserver.handler.count</name>
> <value>50</value>
> </property>
> LOG
> BEGIN----------------------------------------------------------------------
> REGION SERVER ERROR LOG:
> 2012-11-09 00:00:09,312 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:00:09,354 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.67 MB, single=154.11 MB, multi=463.94
> MB, memory=0 KB
> 2012-11-09 00:00:27,770 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.43 MB
> 2012-11-09 00:00:27,781 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.45 MB, total=549.98 MB, single=152.15 MB, multi=465.26
> MB, memory=0 KB
> 2012-11-09 00:00:45,869 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:00:45,880 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.08 MB, single=154.92 MB, multi=462.54
> MB, memory=0 KB
> 2012-11-09 00:00:58,576 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 00:00:58,580 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
> due to global heap pressure
> 2012-11-09 00:00:58,580 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.,
> current region memstore size 104.2m
> 2012-11-09 00:00:58,580 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.,
> commencing wait for mvcc, flushsize=109299760
> 2012-11-09 00:00:58,580 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 00:00:58,625 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524with
> permission:rwxrwxrwx
> 2012-11-09 00:00:58,657 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:00:58,657 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524:
> CompoundBloomFilterWriter
> 2012-11-09 00:00:59,857 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524)
> 2012-11-09 00:00:59,857 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=248607, memsize=104.2m, into tmp file
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524
> 2012-11-09 00:00:59,864 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/c7099a89751b4bedb3e9554569169524
> to
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/c7099a89751b4bedb3e9554569169524
> 2012-11-09 00:00:59,915 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/c7099a89751b4bedb3e9554569169524,
> entries=421945, sequenceid=248607, filesize=32.4m
> 2012-11-09 00:00:59,917 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~104.2m/109299760, currentsize=0.0/0 for region
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
> in 1336ms, sequenceid=248607, compaction requested=true
> 2012-11-09 00:00:59,917 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> 9502c1830ad553d1e85497f9445d30d3 - redirect_info: Initiating minorcompaction
> 2012-11-09 00:00:59,917 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Starting compaction on redirect_info in region
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
> 2012-11-09 00:00:59,917 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small Compaction
> requested:
> regionName=taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.,
> storeName=redirect_info, fileCount=3, fileSize=270.7m (204.6m, 33.6m,
> 32.4m), priority=3, time=696695448849702; Because:
> regionserver60020.cacheFlusher; compaction_queue=(0:0), split_queue=0
> 2012-11-09 00:00:59,918 INFO org.apache.hadoop.hbase.regionserver.Store:
> Starting compaction of 3 file(s) in redirect_info of
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
> into
> tmpdir=hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp,
> seqid=248607, totalSize=270.7m
> 2012-11-09 00:00:59,918 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/95ea533c79b14d70b816136d79402235,
> keycount=2633480, bloomtype=NONE, size=204.6m, encoding=NONE
> 2012-11-09 00:00:59,918 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/9419b66fd8e34977a8b0f48b4aa1ec05,
> keycount=438245, bloomtype=NONE, size=33.6m, encoding=NONE
> 2012-11-09 00:00:59,918 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/c7099a89751b4bedb3e9554569169524,
> keycount=421945, bloomtype=NONE, size=32.4m, encoding=NONE
> 2012-11-09 00:00:59,955 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868ewith
> permission:rwxrwxrwx
> 2012-11-09 00:00:59,989 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:00:59,989 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868e:
> CompoundBloomFilterWriter
> 2012-11-09 00:01:04,115 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:01:04,126 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=549.99 MB, single=148.89 MB, multi=468.51
> MB, memory=0 KB
> 2012-11-09 00:01:09,559 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868e)
> 2012-11-09 00:01:09,566 INFO org.apache.hadoop.hbase.regionserver.Store:
> Renaming compacted file at
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/.tmp/ba824998e14e4a13a309cfa933a4868e
> to
> hdfs://master:54310/hbase/taobao_redirect_table/9502c1830ad553d1e85497f9445d30d3/redirect_info/ba824998e14e4a13a309cfa933a4868e
> 2012-11-09 00:01:09,752 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed compaction of 3 file(s) in redirect_info of
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
> into ba824998e14e4a13a309cfa933a4868e, size=270.7m; total size for store is
> 1.2g
> 2012-11-09 00:01:09,752 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.,
> storeName=redirect_info, fileCount=3, fileSize=270.7m, priority=3,
> time=696695448849702; duration=9sec
> 2012-11-09 00:01:09,752 DEBUG
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> CompactSplitThread status: compaction_queue=(0:0), split_queue=0
> 2012-11-09 00:01:23,517 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.4 MB of total=623.43 MB
> 2012-11-09 00:01:23,528 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.09 MB, single=142.6 MB, multi=474.81
> MB, memory=0 KB
> 2012-11-09 00:01:41,897 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:01:41,908 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=549.97 MB, single=160.06 MB, multi=457.34
> MB, memory=0 KB
> 2012-11-09 00:02:01,739 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:02:01,749 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.25 MB, single=146.18 MB, multi=471.34
> MB, memory=0 KB
> 2012-11-09 00:02:20,779 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:02:20,791 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.05 MB, single=158.69 MB, multi=458.66
> MB, memory=0 KB
> 2012-11-09 00:02:42,214 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:02:42,225 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.09 MB, single=145.46 MB, multi=471.98
> MB, memory=0 KB
> 2012-11-09 00:02:59,501 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:02:59,512 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.05 MB, single=151.19 MB, multi=466.23
> MB, memory=0 KB
> 2012-11-09 00:03:26,921 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.43 MB
> 2012-11-09 00:03:26,931 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.05 MB, single=144.43 MB, multi=472.98
> MB, memory=0 KB
> 2012-11-09 00:04:01,946 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:04:01,957 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.31 MB, single=152.27 MB, multi=465.16
> MB, memory=0 KB
> 2012-11-09 00:04:24,838 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:04:24,848 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.01 MB, single=150.66 MB, multi=466.7
> MB, memory=0 KB
> 2012-11-09 00:04:39,188 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=592.36 MB,
> free=141.01 MB, max=733.38 MB, blocks=9276, accesses=4770844427,
> hits=4766817449, hitRatio=99.91%, cachingAccesses=4770192126,
> cachingHits=4766793835, cachingHitsRatio=99.92%, evictions=2892,
> evicted=3389016, evictedPerRun=1171.85888671875
> 2012-11-09 00:04:46,619 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:04:46,624 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.03 MB, single=161.92 MB, multi=455.46
> MB, memory=0 KB
> 2012-11-09 00:05:04,154 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:05:04,165 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.02 MB, single=143.41 MB, multi=473.96
> MB, memory=0 KB
> 2012-11-09 00:05:20,916 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:05:20,927 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.04 MB, single=158.61 MB, multi=458.83
> MB, memory=0 KB
> 2012-11-09 00:05:37,691 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:05:37,701 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=549.99 MB, single=149.13 MB, multi=468.24
> MB, memory=0 KB
> 2012-11-09 00:05:59,474 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:05:59,485 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=550.23 MB, single=148.26 MB, multi=469.11
> MB, memory=0 KB
> 2012-11-09 00:06:16,659 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.39 MB
> 2012-11-09 00:06:16,670 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=549.98 MB, single=157.47 MB, multi=459.9
> MB, memory=0 KB
> 2012-11-09 00:06:34,145 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:06:34,152 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.09 MB, single=149.44 MB, multi=467.94
> MB, memory=0 KB
> 2012-11-09 00:06:49,302 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:06:49,314 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.3 MB, single=151.87 MB, multi=465.61
> MB, memory=0 KB
> 2012-11-09 00:07:08,734 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.43 MB of total=623.46 MB
> 2012-11-09 00:07:08,748 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.02 MB, single=146.79 MB, multi=470.65
> MB, memory=0 KB
> 2012-11-09 00:07:28,549 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:07:28,560 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.43 MB, total=550 MB, single=146.44 MB, multi=470.97 MB,
> memory=0 KB
> 2012-11-09 00:07:47,367 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:07:47,372 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.13 MB, single=150.1 MB, multi=467.33
> MB, memory=0 KB
> 2012-11-09 00:08:11,442 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:08:11,453 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=550.04 MB, single=148.84 MB, multi=468.53
> MB, memory=0 KB
> 2012-11-09 00:08:31,151 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:08:31,162 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=550.08 MB, single=158.79 MB, multi=458.61
> MB, memory=0 KB
> 2012-11-09 00:08:48,322 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 00:08:48,323 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. due to global
> heap pressure
> 2012-11-09 00:08:48,323 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6., current region
> memstore size 122.6m
> 2012-11-09 00:08:48,323 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6., commencing wait
> for mvcc, flushsize=128508424
> 2012-11-09 00:08:48,323 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 00:08:48,358 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940ewith
> permission:rwxrwxrwx
> 2012-11-09 00:08:48,400 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:08:48,400 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e:
> CompoundBloomFilterWriter
> 2012-11-09 00:08:49,977 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e)
> 2012-11-09 00:08:49,977 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=248709, memsize=122.6m, into tmp file
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e
> 2012-11-09 00:08:49,984 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/571dc1e866184ca6aed3baa2d491940e
> to
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/571dc1e866184ca6aed3baa2d491940e
> 2012-11-09 00:08:50,038 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/571dc1e866184ca6aed3baa2d491940e,
> entries=443292, sequenceid=248709, filesize=47.3m
> 2012-11-09 00:08:50,040 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~122.6m/128508424, currentsize=0.0/0 for region
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. in 1717ms,
> sequenceid=248709, compaction requested=true
> 2012-11-09 00:08:50,040 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> 77c58ab3b4586478b84f4922950d08e6 - record_info: Initiating minorcompaction
> 2012-11-09 00:08:50,040 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small Compaction
> requested:
> regionName=taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.,
> storeName=record_info, fileCount=3, fileSize=133.6m (53.8m, 32.5m, 47.3m),
> priority=2, time=697165571949702; Because: regionserver60020.cacheFlusher;
> compaction_queue=(0:1), split_queue=0
> 2012-11-09 00:08:50,040 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Starting compaction on record_info in region
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
> 2012-11-09 00:08:50,040 INFO org.apache.hadoop.hbase.regionserver.Store:
> Starting compaction of 3 file(s) in record_info of
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. into
> tmpdir=hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp,
> seqid=248709, totalSize=133.6m
> 2012-11-09 00:08:50,040 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/836efce1f8944c79b2cad46cd70bcfd4,
> keycount=507310, bloomtype=NONE, size=53.8m, encoding=NONE
> 2012-11-09 00:08:50,040 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/a914cf73192e4f52a06cbac1f62d92d1,
> keycount=313518, bloomtype=NONE, size=32.5m, encoding=NONE
> 2012-11-09 00:08:50,041 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/571dc1e866184ca6aed3baa2d491940e,
> keycount=443292, bloomtype=NONE, size=47.3m, encoding=NONE
> 2012-11-09 00:08:50,076 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44with
> permission:rwxrwxrwx
> 2012-11-09 00:08:50,110 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:08:50,110 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44:
> CompoundBloomFilterWriter
> 2012-11-09 00:08:50,815 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:08:50,829 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.35 MB, total=550.21 MB, single=153.5 MB, multi=464.05
> MB, memory=0 KB
> 2012-11-09 00:08:52,225 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. with WAL
> disabled. Data may be lost in the event of a crash.
> 2012-11-09 00:08:54,736 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44)
> 2012-11-09 00:08:54,744 INFO org.apache.hadoop.hbase.regionserver.Store:
> Renaming compacted file at
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/.tmp/dbe372eda2fb4b76aa518af03f7bff44
> to
> hdfs://master:54310/hbase/taobao_record_table/77c58ab3b4586478b84f4922950d08e6/record_info/dbe372eda2fb4b76aa518af03f7bff44
> 2012-11-09 00:08:54,920 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed compaction of 3 file(s) in record_info of
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6. into
> dbe372eda2fb4b76aa518af03f7bff44, size=129.0m; total size for store is 1.6g
> 2012-11-09 00:08:54,920 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.,
> storeName=record_info, fileCount=3, fileSize=133.6m, priority=2,
> time=697165571949702; duration=4sec
> 2012-11-09 00:08:54,920 DEBUG
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> CompactSplitThread status: compaction_queue=(0:0), split_queue=0
> 2012-11-09 00:09:11,150 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:09:11,155 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=550.11 MB, single=148.66 MB, multi=468.77
> MB, memory=0 KB
> 2012-11-09 00:09:28,449 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:09:28,460 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.09 MB, single=153.04 MB, multi=464.41
> MB, memory=0 KB
> 2012-11-09 00:09:39,194 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=591.8 MB,
> free=141.58 MB, max=733.38 MB, blocks=9267, accesses=4794675147,
> hits=4790627441, hitRatio=99.91%, cachingAccesses=4794020702,
> cachingHits=4790603814, cachingHitsRatio=99.92%, evictions=2908,
> evicted=3407621, evictedPerRun=1171.8092041015625
> 2012-11-09 00:09:49,245 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:09:49,255 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.07 MB, single=149.17 MB, multi=468.28
> MB, memory=0 KB
> 2012-11-09 00:10:12,186 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:10:12,191 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.18 MB, single=153.84 MB, multi=463.59
> MB, memory=0 KB
> 2012-11-09 00:10:29,793 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.39 MB
> 2012-11-09 00:10:29,804 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.01 MB, single=159.33 MB, multi=458.03
> MB, memory=0 KB
> 2012-11-09 00:10:49,676 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:10:49,683 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.43 MB, total=550.04 MB, single=144.88 MB, multi=472.51
> MB, memory=0 KB
> 2012-11-09 00:11:09,801 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:11:09,813 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.14 MB, single=156.23 MB, multi=461.17
> MB, memory=0 KB
> 2012-11-09 00:11:39,395 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.41 MB
> 2012-11-09 00:11:39,405 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.11 MB, single=147.77 MB, multi=469.61
> MB, memory=0 KB
> 2012-11-09 00:12:02,520 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:12:02,529 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.1 MB, single=161.43 MB, multi=455.99
> MB, memory=0 KB
> 2012-11-09 00:12:22,728 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:12:22,734 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.02 MB, single=142.63 MB, multi=474.76
> MB, memory=0 KB
> 2012-11-09 00:12:50,996 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:12:51,007 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.43 MB, total=550.1 MB, single=156.5 MB, multi=460.88
> MB, memory=0 KB
> 2012-11-09 00:13:09,661 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:13:09,673 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.31 MB, single=157.14 MB, multi=460.36
> MB, memory=0 KB
> 2012-11-09 00:13:28,194 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:13:28,205 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.13 MB, single=148.29 MB, multi=469.07
> MB, memory=0 KB
> 2012-11-09 00:13:46,681 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:13:46,692 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.2 MB, single=153.92 MB, multi=463.58
> MB, memory=0 KB
> 2012-11-09 00:14:06,839 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:14:06,850 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=550.13 MB, single=148.52 MB, multi=468.94
> MB, memory=0 KB
> 2012-11-09 00:14:26,835 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:14:26,846 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.04 MB, single=149.82 MB, multi=467.54
> MB, memory=0 KB
> 2012-11-09 00:14:39,187 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=589.1 MB,
> free=144.28 MB, max=733.38 MB, blocks=9226, accesses=4817256651,
> hits=4813192706, hitRatio=99.91%, cachingAccesses=4816602220,
> cachingHits=4813169094, cachingHitsRatio=99.92%, evictions=2922,
> evicted=3423901, evictedPerRun=1171.7662353515625
> 2012-11-09 00:14:48,384 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:14:48,395 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.19 MB, single=150.69 MB, multi=466.84
> MB, memory=0 KB
> 2012-11-09 00:15:06,866 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:15:06,871 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.09 MB, single=157.06 MB, multi=460.39
> MB, memory=0 KB
> 2012-11-09 00:15:26,386 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:15:26,391 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.03 MB, single=142.63 MB, multi=474.77
> MB, memory=0 KB
> 2012-11-09 00:15:46,206 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:15:46,212 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.06 MB, single=155.44 MB, multi=461.95
> MB, memory=0 KB
> 2012-11-09 00:16:10,667 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:16:10,677 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.02 MB, single=151.52 MB, multi=465.86
> MB, memory=0 KB
> 2012-11-09 00:16:43,503 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 00:16:43,503 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> due to global heap pressure
> 2012-11-09 00:16:43,503 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.,
> current region memstore size 110.6m
> 2012-11-09 00:16:43,504 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.,
> commencing wait for mvcc, flushsize=115944576
> 2012-11-09 00:16:43,504 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 00:16:43,548 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> with WAL disabled. Data may be lost in the event of a crash.
> 2012-11-09 00:16:43,561 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91dwith
> permission:rwxrwxrwx
> 2012-11-09 00:16:43,622 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:16:43,622 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d:
> CompoundBloomFilterWriter
> 2012-11-09 00:16:44,524 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d)
> 2012-11-09 00:16:44,524 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=248865, memsize=87.5m, into tmp file
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d
> 2012-11-09 00:16:44,530 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/c66335aaedee4ed48ce08d2ed32ab91d
> to
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/c66335aaedee4ed48ce08d2ed32ab91d
> 2012-11-09 00:16:44,580 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/c66335aaedee4ed48ce08d2ed32ab91d,
> entries=356415, sequenceid=248865, filesize=27.0m
> 2012-11-09 00:16:44,581 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~110.6m/115944576, currentsize=57.9k/59328 for
> region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> in 1078ms, sequenceid=248865, compaction requested=false
> 2012-11-09 00:16:54,437 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.4 MB of total=623.43 MB
> 2012-11-09 00:16:54,447 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.45 MB, total=550.1 MB, single=153.49 MB, multi=463.92
> MB, memory=0 KB
> 2012-11-09 00:17:15,232 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:17:15,243 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.18 MB, single=156.88 MB, multi=460.54
> MB, memory=0 KB
> 2012-11-09 00:17:31,641 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:17:31,649 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.03 MB, single=155.66 MB, multi=461.73
> MB, memory=0 KB
> 2012-11-09 00:17:55,192 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.43 MB
> 2012-11-09 00:17:55,197 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.22 MB, single=151.95 MB, multi=465.52
> MB, memory=0 KB
> 2012-11-09 00:18:14,279 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:18:14,289 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.12 MB, single=146.87 MB, multi=470.56
> MB, memory=0 KB
> 2012-11-09 00:18:31,892 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:18:31,903 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.11 MB, single=148.45 MB, multi=468.95
> MB, memory=0 KB
> 2012-11-09 00:18:55,244 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:18:55,249 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.02 MB, single=149.58 MB, multi=467.82
> MB, memory=0 KB
> 2012-11-09 00:19:14,647 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:19:14,652 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.12 MB, single=161.91 MB, multi=455.53
> MB, memory=0 KB
> 2012-11-09 00:19:33,690 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:19:33,701 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.14 MB, single=146.9 MB, multi=470.48
> MB, memory=0 KB
> 2012-11-09 00:19:39,193 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=573.16 MB,
> free=160.21 MB, max=733.38 MB, blocks=8974, accesses=4837602264,
> hits=4833522292, hitRatio=99.91%, cachingAccesses=4836947819,
> cachingHits=4833498665, cachingHitsRatio=99.92%, evictions=2936,
> evicted=3440181, evictedPerRun=1171.7237548828125
> 2012-11-09 00:19:54,689 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:19:54,700 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=550.1 MB, single=143.48 MB, multi=473.95
> MB, memory=0 KB
> 2012-11-09 00:20:17,993 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:20:18,004 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.02 MB, single=152.53 MB, multi=464.87
> MB, memory=0 KB
> 2012-11-09 00:20:56,068 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:20:56,079 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550 MB, single=153.66 MB, multi=463.69 MB,
> memory=0 KB
> 2012-11-09 00:21:20,070 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:21:20,081 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550 MB, single=150.06 MB, multi=467.29 MB,
> memory=0 KB
> 2012-11-09 00:21:39,357 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:21:39,368 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=549.97 MB, single=148.17 MB, multi=469.22
> MB, memory=0 KB
> 2012-11-09 00:21:53,786 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.41 MB
> 2012-11-09 00:21:53,791 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.16 MB, single=153.64 MB, multi=463.75
> MB, memory=0 KB
> 2012-11-09 00:22:09,045 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:22:09,050 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=550.19 MB, single=142.22 MB, multi=475.23
> MB, memory=0 KB
> 2012-11-09 00:22:22,680 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:22:22,685 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.4 MB, total=549.99 MB, single=154.99 MB, multi=462.38
> MB, memory=0 KB
> 2012-11-09 00:22:36,247 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.43 MB
> 2012-11-09 00:22:36,259 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.11 MB, single=151.06 MB, multi=466.42
> MB, memory=0 KB
> 2012-11-09 00:22:47,994 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:22:48,006 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.11 MB, single=146.43 MB, multi=471.03
> MB, memory=0 KB
> 2012-11-09 00:22:57,536 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.43 MB
> 2012-11-09 00:22:57,542 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.13 MB, single=138.95 MB, multi=478.52
> MB, memory=0 KB
> 2012-11-09 00:23:07,835 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:23:07,845 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.06 MB, single=154.92 MB, multi=462.52
> MB, memory=0 KB
> 2012-11-09 00:23:17,648 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.34 MB of total=623.37 MB
> 2012-11-09 00:23:17,659 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.38 MB, single=148.53 MB, multi=468.95
> MB, memory=0 KB
> 2012-11-09 00:23:30,995 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:23:31,007 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.09 MB, single=142.48 MB, multi=474.91
> MB, memory=0 KB
> 2012-11-09 00:23:46,183 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:23:46,194 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.02 MB, single=154.22 MB, multi=463.17
> MB, memory=0 KB
> 2012-11-09 00:24:06,894 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:24:06,904 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.41 MB, total=549.98 MB, single=146.89 MB, multi=470.49
> MB, memory=0 KB
> 2012-11-09 00:24:12,173 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 00:24:12,174 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> due to global heap pressure
> 2012-11-09 00:24:12,174 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.,
> current region memstore size 114.6m
> 2012-11-09 00:24:12,174 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.,
> commencing wait for mvcc, flushsize=120129544
> 2012-11-09 00:24:12,174 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 00:24:12,257 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684with
> permission:rwxrwxrwx
> 2012-11-09 00:24:12,304 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:24:12,304 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684:
> CompoundBloomFilterWriter
> 2012-11-09 00:24:12,432 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> with WAL disabled. Data may be lost in the event of a crash.
> 2012-11-09 00:24:13,179 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684)
> 2012-11-09 00:24:13,180 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=248944, memsize=80.3m, into tmp file
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684
> 2012-11-09 00:24:13,186 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/a77e1905f21a4174b594d6f8d64cd684
> to
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/a77e1905f21a4174b594d6f8d64cd684
> 2012-11-09 00:24:13,238 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/a77e1905f21a4174b594d6f8d64cd684,
> entries=326993, sequenceid=248944, filesize=24.8m
> 2012-11-09 00:24:13,239 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~114.6m/120129544, currentsize=37.1k/37960 for
> region
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> in 1065ms, sequenceid=248944, compaction requested=false
> 2012-11-09 00:24:26,978 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.42 MB
> 2012-11-09 00:24:26,989 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.29 MB, single=154.78 MB, multi=462.69
> MB, memory=0 KB
> 2012-11-09 00:24:39,188 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=598.3 MB,
> free=135.08 MB, max=733.38 MB, blocks=9373, accesses=4865823971,
> hits=4861723825, hitRatio=99.91%, cachingAccesses=4865169535,
> cachingHits=4861700208, cachingHitsRatio=99.92%, evictions=2953,
> evicted=3459954, evictedPerRun=1171.6741943359375
> 2012-11-09 00:24:45,922 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:24:45,934 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.09 MB, single=147.02 MB, multi=470.38
> MB, memory=0 KB
> 2012-11-09 00:25:04,214 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:25:04,225 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.21 MB, single=151.87 MB, multi=465.57
> MB, memory=0 KB
> 2012-11-09 00:25:29,735 DEBUG
> org.apache.hadoop.hbase.regionserver.LogRoller: Hlog roll period 3600000ms
> elapsed
> 2012-11-09 00:25:29,788 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: using new
> createWriter -- HADOOP-6840
> 2012-11-09 00:25:29,789 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter:
> Path=hdfs://master:54310/hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352391929736,
> syncFs=true, hflush=false, compression=false
> 2012-11-09 00:25:29,840 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: Roll
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352388328705,
> entries=1867, filesize=4237942. for
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352391929736
> 2012-11-09 00:25:29,841 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.HLog: Found 9 hlogs to remove out
> of total 22; oldest outstanding sequenceid is 181222 from region
> 5b2ca166dc37841bdecb51e4cb942914
> 2012-11-09 00:25:29,841 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352309103831
> whose highest sequenceid is 177189 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352309103831
> 2012-11-09 00:25:29,922 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352312704819
> whose highest sequenceid is 178134 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352312704819
> 2012-11-09 00:25:29,964 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352316305787
> whose highest sequenceid is 178676 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352316305787
> 2012-11-09 00:25:30,006 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352319907042
> whose highest sequenceid is 179352 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352319907042
> 2012-11-09 00:25:30,047 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352323508031
> whose highest sequenceid is 179994 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352323508031
> 2012-11-09 00:25:30,113 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352327108994
> whose highest sequenceid is 180535 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352327108994
> 2012-11-09 00:25:30,163 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352330709960
> whose highest sequenceid is 180912 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352330709960
> 2012-11-09 00:25:30,198 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352334310944
> whose highest sequenceid is 181137 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352334310944
> 2012-11-09 00:25:30,248 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: moving old hlog file
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352337912037
> whose highest sequenceid is 181221 to
> /hbase/.oldlogs/slave181%2C60020%2C1352287478815.1352337912037
> 2012-11-09 00:25:37,177 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:25:37,183 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.16 MB, single=155.91 MB, multi=461.48
> MB, memory=0 KB
> 2012-11-09 00:25:53,366 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:25:53,377 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.39 MB, total=550.06 MB, single=150.85 MB, multi=466.52
> MB, memory=0 KB
> 2012-11-09 00:26:10,477 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.4 MB of total=623.43 MB
> 2012-11-09 00:26:10,484 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.05 MB, single=150.31 MB, multi=467.17
> MB, memory=0 KB
> 2012-11-09 00:26:29,939 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:26:29,946 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=549.99 MB, single=152 MB, multi=465.41 MB,
> memory=0 KB
> 2012-11-09 00:26:49,262 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:26:49,271 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.1 MB, single=149.6 MB, multi=467.8 MB,
> memory=0 KB
> 2012-11-09 00:27:03,160 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:27:03,172 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.11 MB, single=152.89 MB, multi=464.64
> MB, memory=0 KB
> 2012-11-09 00:27:27,736 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:27:27,747 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.43 MB, total=549.99 MB, single=143.24 MB, multi=474.17
> MB, memory=0 KB
> 2012-11-09 00:27:47,871 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.4 MB of total=623.43 MB
> 2012-11-09 00:27:47,883 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.14 MB, single=148.23 MB, multi=469.25
> MB, memory=0 KB
> 2012-11-09 00:28:14,121 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.36 MB of total=623.39 MB
> 2012-11-09 00:28:14,126 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.07 MB, single=150.18 MB, multi=467.25
> MB, memory=0 KB
> 2012-11-09 00:28:38,799 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:28:38,811 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.06 MB, single=149.8 MB, multi=467.56
> MB, memory=0 KB
> 2012-11-09 00:29:09,256 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:29:09,267 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.16 MB, single=148.07 MB, multi=469.39
> MB, memory=0 KB
> 2012-11-09 00:29:25,085 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:29:25,097 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.04 MB, single=158.04 MB, multi=459.36
> MB, memory=0 KB
> 2012-11-09 00:29:39,194 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=601.77 MB,
> free=131.6 MB, max=733.38 MB, blocks=9429, accesses=4887212513,
> hits=4883096023, hitRatio=99.91%, cachingAccesses=4886558080,
> cachingHits=4883072411, cachingHitsRatio=99.92%, evictions=2967,
> evicted=3476243, evictedPerRun=1171.6356201171875
> 2012-11-09 00:29:46,796 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:29:46,802 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.1 MB, single=143.15 MB, multi=474.25
> MB, memory=0 KB
> 2012-11-09 00:30:03,306 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:30:03,317 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.22 MB, single=158.23 MB, multi=459.17
> MB, memory=0 KB
> 2012-11-09 00:30:21,541 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:30:21,547 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.03 MB, single=147.05 MB, multi=470.33
> MB, memory=0 KB
> 2012-11-09 00:30:40,859 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:30:40,864 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.03 MB, single=150.52 MB, multi=466.86
> MB, memory=0 KB
> 2012-11-09 00:31:00,139 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:31:00,149 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.1 MB, single=152.87 MB, multi=464.53
> MB, memory=0 KB
> 2012-11-09 00:31:09,372 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 00:31:09,372 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3.
> due to global heap pressure
> 2012-11-09 00:31:09,372 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3.,
> current region memstore size 113.7m
> 2012-11-09 00:31:09,372 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3.,
> commencing wait for mvcc, flushsize=119262296
> 2012-11-09 00:31:09,373 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 00:31:09,420 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541with
> permission:rwxrwxrwx
> 2012-11-09 00:31:09,461 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:31:09,461 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541:
> CompoundBloomFilterWriter
> 2012-11-09 00:31:10,911 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541)
> 2012-11-09 00:31:10,911 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=249027, memsize=113.7m, into tmp file
> hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541
> 2012-11-09 00:31:10,918 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/.tmp/bd3a73218c964f88b9a3c3878d9ba541
> to
> hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/item_info/bd3a73218c964f88b9a3c3878d9ba541
> 2012-11-09 00:31:10,965 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3.
> with WAL disabled. Data may be lost in the event of a crash.
> 2012-11-09 00:31:10,977 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_item_his_table/278ef97228af5cf2ccf547ef332450d3/item_info/bd3a73218c964f88b9a3c3878d9ba541,
> entries=422702, sequenceid=249027, filesize=41.9m
> 2012-11-09 00:31:10,980 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~113.7m/119262296, currentsize=21.9k/22456 for
> region
> taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3.
> in 1608ms, sequenceid=249027, compaction requested=false
> 2012-11-09 00:31:18,694 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:31:18,705 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.04 MB, single=149.53 MB, multi=467.93
> MB, memory=0 KB
> 2012-11-09 00:31:36,194 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:31:36,205 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.1 MB, single=147.08 MB, multi=470.31
> MB, memory=0 KB
> 2012-11-09 00:31:59,393 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 00:31:59,398 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.02 MB, single=147.63 MB, multi=469.75
> MB, memory=0 KB
> 2012-11-09 00:32:20,362 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.4 MB of total=623.43 MB
> 2012-11-09 00:32:20,373 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.18 MB, single=154.1 MB, multi=463.44
> MB, memory=0 KB
> 2012-11-09 00:32:41,480 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> 2012-11-09 00:32:41,491 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=549.99 MB, single=151 MB, multi=466.41 MB,
> memory=0 KB
> 2012-11-09 00:33:03,782 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.39 MB of total=623.42 MB
> ....
> 2012-11-09 00:39:39,194 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=585.46 MB,
> free=147.91 MB, max=733.38 MB, blocks=9173, accesses=4935896422,
> hits=4931744129, hitRatio=99.91%, cachingAccesses=4935241977,
> cachingHits=4931720502, cachingHitsRatio=99.92%, evictions=2998,
> evicted=3512303, evictedPerRun=1171.5487060546875
> 2012-11-09 00:39:49,782 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:39:49,793 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.19 MB, single=153.7 MB, multi=463.79
> MB, memory=0 KB
> 2012-11-09 00:40:09,319 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:40:09,329 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.07 MB, single=151.11 MB, multi=466.25
> MB, memory=0 KB
> 2012-11-09 00:40:26,066 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> 2012-11-09 00:40:26,077 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.37 MB, total=550.38 MB, single=156.6 MB, multi=460.83
> MB, memory=0 KB
> 2012-11-09 00:40:33,396 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 00:40:33,396 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> due to global heap pressure
> 2012-11-09 00:40:33,396 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.,
> current region memstore size 137.5m
> 2012-11-09 00:40:33,396 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.,
> commencing wait for mvcc, flushsize=144184360
> 2012-11-09 00:40:33,396 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 00:40:33,444 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> with WAL disabled. Data may be lost in the event of a crash.
> 2012-11-09 00:40:33,481 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94with
> permission:rwxrwxrwx
> 2012-11-09 00:40:33,516 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:40:33,516 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94:
> CompoundBloomFilterWriter
> 2012-11-09 00:40:34,548 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94)
> 2012-11-09 00:40:34,548 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=249054, memsize=89.6m, into tmp file
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94
> 2012-11-09 00:40:34,553 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/f94b49bad2884cf38b19bf173e314e94
> to
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/f94b49bad2884cf38b19bf173e314e94
> 2012-11-09 00:40:34,612 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/f94b49bad2884cf38b19bf173e314e94,
> entries=364696, sequenceid=249054, filesize=27.7m
> 2012-11-09 00:40:34,613 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~137.5m/144184360, currentsize=65.2k/66816 for
> region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> in 1217ms, sequenceid=249054, compaction requested=false
> 2012-11-09 00:40:46,468 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.35 MB of total=623.38 MB
> .....
> 2012-11-09 00:51:20,609 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.05 MB, single=151.84 MB, multi=465.54
> MB, memory=0 KB
> 2012-11-09 00:51:59,887 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:51:59,898 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.1 MB, single=150.96 MB, multi=466.43
> MB, memory=0 KB
> 2012-11-09 00:52:47,163 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:52:47,174 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.1 MB, single=153.11 MB, multi=464.35
> MB, memory=0 KB
> 2012-11-09 00:52:53,378 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 00:52:53,379 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> due to global heap pressure
> 2012-11-09 00:52:53,379 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.,
> current region memstore size 126.8m
> 2012-11-09 00:52:53,379 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.,
> commencing wait for mvcc, flushsize=132907720
> 2012-11-09 00:52:53,379 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 00:52:53,385 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> with WAL disabled. Data may be lost in the event of a crash.
> 2012-11-09 00:52:53,452 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217with
> permission:rwxrwxrwx
> 2012-11-09 00:52:53,486 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 00:52:53,486 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217:
> CompoundBloomFilterWriter
> 2012-11-09 00:52:54,378 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217)
> 2012-11-09 00:52:54,378 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=249097, memsize=89.1m, into tmp file
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217
> 2012-11-09 00:52:54,385 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/.tmp/f3cb0947202d40909ac731edfcaac217
> to
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/f3cb0947202d40909ac731edfcaac217
> 2012-11-09 00:52:54,435 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_item_redirect_table/18d2c63be7451b8e12b368552910bb20/item_info/f3cb0947202d40909ac731edfcaac217,
> entries=363251, sequenceid=249097, filesize=27.5m
> 2012-11-09 00:52:54,438 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~126.8m/132907720, currentsize=74.0k/75760 for
> region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> in 1058ms, sequenceid=249097, compaction requested=false
> 2012-11-09 00:53:28,007 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:53:28,018 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.44 MB, total=550.03 MB, single=154.49 MB, multi=462.96
> MB, memory=0 KB
> 2012-11-09 00:54:08,504 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.38 MB of total=623.41 MB
> 2012-11-09 00:54:08,515 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.38 MB, total=550.09 MB, single=148.56 MB, multi=468.89
> MB, memory=0 KB
> 2012-11-09 00:54:39,180 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=596.52 MB,
> free=136.85 MB, max=733.38 MB, blocks=9346, accesses=4984056793,
> hits=4979865932, hitRatio=99.91%, cachingAccesses=4983402348,
> cachingHits=4979842305, cachingHitsRatio=99.92%, evictions=3031,
> evicted=3550697, evictedPerRun=1171.4605712890625
> 2012-11-09 00:54:56,875 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> ......
> 2012-11-09 01:02:37,188 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.43 MB, total=549.98 MB, single=151.59 MB, multi=465.81
> MB, memory=0 KB
> 2012-11-09 01:03:12,945 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 01:03:12,946 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> due to global heap pressure
> 2012-11-09 01:03:12,946 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.,
> current region memstore size 103.3m
> 2012-11-09 01:03:12,946 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.,
> commencing wait for mvcc, flushsize=108273864
> 2012-11-09 01:03:12,946 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 01:03:13,014 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2with
> permission:rwxrwxrwx
> 2012-11-09 01:03:13,047 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 01:03:13,047 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2:
> CompoundBloomFilterWriter
> 2012-11-09 01:03:13,155 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> with WAL disabled. Data may be lost in the event of a crash.
> 2012-11-09 01:03:13,754 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2)
> 2012-11-09 01:03:13,754 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=249293, memsize=67.4m, into tmp file
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2
> 2012-11-09 01:03:13,760 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/54db1c93a33b46ba9e0c06c06a894ff2
> to
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/54db1c93a33b46ba9e0c06c06a894ff2
> 2012-11-09 01:03:13,810 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/54db1c93a33b46ba9e0c06c06a894ff2,
> entries=274258, sequenceid=249293, filesize=20.8m
> 2012-11-09 01:03:13,812 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~103.3m/108273864, currentsize=17.7k/18144 for
> region
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> in 866ms, sequenceid=249293, compaction requested=true
> 2012-11-09 01:03:13,812 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> 1469bafdb43e6d2eea476b9619921d38 - item_info: Initiating minorcompaction
> 2012-11-09 01:03:13,813 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Starting compaction on item_info in region
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> 2012-11-09 01:03:13,813 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small Compaction
> requested:
> regionName=taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.,
> storeName=item_info, fileCount=4, fileSize=317.9m (251.4m, 20.9m, 24.8m,
> 20.8m), priority=3, time=700429344373702; Because:
> regionserver60020.cacheFlusher; compaction_queue=(0:0), split_queue=0
> 2012-11-09 01:03:13,813 INFO org.apache.hadoop.hbase.regionserver.Store:
> Starting compaction of 4 file(s) in item_info of
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> into
> tmpdir=hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp,
> seqid=249293, totalSize=317.9m
> 2012-11-09 01:03:13,813 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/70ffe3f371e84c94b021cf087f6e9df6,
> keycount=3422770, bloomtype=NONE, size=251.4m, encoding=NONE
> 2012-11-09 01:03:13,813 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/6974289ad68e40ce81fd73cef25b20ab,
> keycount=274512, bloomtype=NONE, size=20.9m, encoding=NONE
> 2012-11-09 01:03:13,813 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/a77e1905f21a4174b594d6f8d64cd684,
> keycount=326993, bloomtype=NONE, size=24.8m, encoding=NONE
> 2012-11-09 01:03:13,814 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Compacting
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/54db1c93a33b46ba9e0c06c06a894ff2,
> keycount=274258, bloomtype=NONE, size=20.8m, encoding=NONE
> 2012-11-09 01:03:13,829 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2cwith
> permission:rwxrwxrwx
> 2012-11-09 01:03:13,910 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 01:03:13,911 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2c:
> CompoundBloomFilterWriter
> 2012-11-09 01:03:19,620 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free 73.37 MB of total=623.4 MB
> 2012-11-09 01:03:19,631 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> completed; freed=73.42 MB, total=550.24 MB, single=125.78 MB, multi=491.61
> MB, memory=0 KB
> 2012-11-09 01:03:22,465 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2c)
> 2012-11-09 01:03:22,471 INFO org.apache.hadoop.hbase.regionserver.Store:
> Renaming compacted file at
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/.tmp/585b2b294a5245df88230e666c624b2c
> to
> hdfs://master:54310/hbase/taobao_item_redirect_table/1469bafdb43e6d2eea476b9619921d38/item_info/585b2b294a5245df88230e666c624b2c
> 2012-11-09 01:03:22,722 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 4 file(s) in item_info of
> taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.
> into 585b2b294a5245df88230e666c624b2c, size=255.5m; total size for store is
> 255.5m
> 2012-11-09 01:03:22,722 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=taobao_item_redirect_table,15368767077,1352117930928.1469bafdb43e6d2eea476b9619921d38.,
> storeName=item_info, fileCount=4, fileSize=317.9m, priority=3,
> time=700429344373702; duration=8sec
> 2012-11-09 01:03:22,722 DEBUG
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> CompactSplitThread status: compaction_queue=(0:0), split_queue=0
> .................
> 2012-11-09 01:24:39,187 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=483.42 MB,
> free=249.95 MB, max=733.38 MB, blocks=7555, accesses=5003183182,
> hits=4998959719, hitRatio=99.91%, cachingAccesses=5002509512,
> cachingHits=4998933235, cachingHitsRatio=99.92%, evictions=3044,
> evicted=3568722, evictedPerRun=1172.379150390625
> 2012-11-09 01:25:31,199 DEBUG
> org.apache.hadoop.hbase.regionserver.LogRoller: Hlog roll period 3600000ms
> elapsed
> 2012-11-09 01:25:31,253 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: using new
> createWriter -- HADOOP-6840
> 2012-11-09 01:25:31,254 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter:
> Path=hdfs://master:54310/hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352395531200,
> syncFs=true, hflush=false, compression=false
> 2012-11-09 01:25:31,302 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: Roll
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352391929736,
> entries=632, filesize=415028. for
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352395531200
> 2012-11-09 01:29:39,193 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=488.76 MB,
> free=244.62 MB, max=733.38 MB, blocks=7639, accesses=5003187645,
> hits=4998964098, hitRatio=99.91%, cachingAccesses=5002513975,
> cachingHits=4998937614, cachingHitsRatio=99.92%, evictions=3044,
> evicted=3568722, evictedPerRun=1172.379150390625
> 2012-11-09 01:34:39,187 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=494.67 MB,
> free=238.7 MB, max=733.38 MB, blocks=7732, accesses=5003192070,
> hits=4998968430, hitRatio=99.91%, cachingAccesses=5002518400,
> cachingHits=4998941946, cachingHitsRatio=99.92%, evictions=3044,
> evicted=3568722, evictedPerRun=1172.379150390625
> 2012-11-09 01:37:32,605 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=1.0g
> 2012-11-09 01:37:32,605 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> due to global heap pressure
> 2012-11-09 01:37:32,606 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.,
> current region memstore size 157.8m
> 2012-11-09 01:37:32,606 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.,
> commencing wait for mvcc, flushsize=165513280
> 2012-11-09 01:37:32,606 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting,
> commencing flushing stores
> 2012-11-09 01:37:32,683 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
> file:hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8with
> permission:rwxrwxrwx
> 2012-11-09 01:37:32,720 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> writing data to region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> with WAL disabled. Data may be lost in the event of a crash.
> 2012-11-09 01:37:32,731 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-11-09 01:37:32,731 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8:
> CompoundBloomFilterWriter
> 2012-11-09 01:37:33,707 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
> (hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8)
> 2012-11-09 01:37:33,708 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=249835, memsize=101.6m, into tmp file
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8
> 2012-11-09 01:37:33,714 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/.tmp/50dc3ad378ef45068d8af6e53e6951a8
> to
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/50dc3ad378ef45068d8af6e53e6951a8
> 2012-11-09 01:37:33,763 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
> hdfs://master:54310/hbase/taobao_item_redirect_table/9249f776d3aa6a903869dbeb6aeef49f/item_info/50dc3ad378ef45068d8af6e53e6951a8,
> entries=413606, sequenceid=249835, filesize=31.4m
> 2012-11-09 01:37:33,765 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~157.8m/165513280, currentsize=47.6k/48696 for
> region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> in 1159ms, sequenceid=249835, compaction requested=false
> 2012-11-09 01:39:39,194 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=499.61 MB,
> free=233.76 MB, max=733.38 MB, blocks=7810, accesses=5003196649,
> hits=4998972931, hitRatio=99.91%, cachingAccesses=5002522979,
> cachingHits=4998946447, cachingHitsRatio=99.92%, evictions=3044,
> evicted=3568722, evictedPerRun=1172.379150390625
> 2012-11-09 01:43:12,097 INFO org.apache.zookeeper.ClientCnxn: Client
> session timed out, have not heard from server in 40000ms for sessionid
> 0x13ada9d738a0001, closing socket connection and attempting reconnect
> 2012-11-09 01:43:12,204 INFO org.apache.zookeeper.ClientCnxn: Client
> session timed out, have not heard from server in 40000ms for sessionid
> 0x13ada9d738a000b, closing socket connection and attempting reconnect
> 2012-11-09 01:43:12,897 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server /192.168.2.180:2222
> 2012-11-09 01:43:12,897 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:13,042 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave180/192.168.2.180:2222
> 2012-11-09 01:43:13,042 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:21,468 INFO org.apache.zookeeper.ClientCnxn: Client
> session timed out, have not heard from server in 9162ms for sessionid
> 0x13ada9d738a000b, closing socket connection and attempting reconnect
> 2012-11-09 01:43:21,613 INFO org.apache.zookeeper.ClientCnxn: Client
> session timed out, have not heard from server in 9414ms for sessionid
> 0x13ada9d738a0001, closing socket connection and attempting reconnect
> 2012-11-09 01:43:22,283 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave182/192.168.2.182:2222
> 2012-11-09 01:43:22,283 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:22,572 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server /192.168.2.182:2222
> 2012-11-09 01:43:22,572 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:22,573 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a000b for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:23,301 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave184/192.168.2.184:2222
> 2012-11-09 01:43:23,301 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:23,302 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a000b for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:24,142 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave185/192.168.2.185:2222
> 2012-11-09 01:43:24,142 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:24,143 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a000b for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:25,146 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave183/192.168.2.183:2222
> 2012-11-09 01:43:25,146 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:25,147 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a000b for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:25,283 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:25,613 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave181/192.168.2.181:2222
> 2012-11-09 01:43:25,613 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:25,614 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:25,782 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave181/192.168.2.181:2222
> 2012-11-09 01:43:25,782 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:25,783 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a000b for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:26,269 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave185/192.168.2.185:2222
> 2012-11-09 01:43:26,269 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:26,270 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:26,529 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave184/192.168.2.184:2222
> 2012-11-09 01:43:26,529 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:26,529 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:27,072 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server master/192.168.2.167:2222
> 2012-11-09 01:43:27,072 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:27,073 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to master/192.168.2.167:2222, initiating session
> 2012-11-09 01:43:27,093 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server master/192.168.2.167:2222, sessionid =
> 0x13ada9d738a000b, negotiated timeout = 60000
> 2012-11-09 01:43:27,584 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave183/192.168.2.183:2222
> 2012-11-09 01:43:27,584 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:36,155 INFO org.apache.zookeeper.ClientCnxn: Client
> session timed out, have not heard from server in 9523ms for sessionid
> 0x13ada9d738a0001, closing socket connection and attempting reconnect
> 2012-11-09 01:43:38,109 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server master/192.168.2.167:2222
> 2012-11-09 01:43:38,109 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:46,680 INFO org.apache.zookeeper.ClientCnxn: Client
> session timed out, have not heard from server in 10423ms for sessionid
> 0x13ada9d738a0001, closing socket connection and attempting reconnect
> 2012-11-09 01:43:47,537 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave180/192.168.2.180:2222
> 2012-11-09 01:43:47,537 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:49,126 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: master/192.168.2.167:54310. Already tried 0 time(s).
> 2012-11-09 01:43:50,537 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:50,979 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave182/192.168.2.182:2222
> 2012-11-09 01:43:50,979 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:50,980 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:51,234 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave181/192.168.2.181:2222
> 2012-11-09 01:43:51,234 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:51,235 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:51,452 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave185/192.168.2.185:2222
> 2012-11-09 01:43:51,452 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:51,452 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:52,221 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave184/192.168.2.184:2222
> 2012-11-09 01:43:52,221 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:52,222 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:52,352 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server slave183/192.168.2.183:2222
> 2012-11-09 01:43:52,352 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:52,352 WARN org.apache.zookeeper.ClientCnxn: Session
> 0x13ada9d738a0001 for server null, unexpected error, closing socket
> connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 2012-11-09 01:43:53,358 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> slave181,60020,1352287478815: Unhandled exception:
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently processing slave181,60020,1352287478815 as dead server
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently processing slave181,60020,1352287478815 as dead server
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:852)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:723)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently processing slave181,60020,1352287478815 as dead server
> at
> org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:242)
> at
> org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:160)
> at
> org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:935)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1400)
>
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1021)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> at $Proxy8.regionServerReport(Unknown Source)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:849)
> ... 2 more
> 2012-11-09 01:43:53,374 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort:
> loaded coprocessors are: []
> 2012-11-09 01:43:53,378 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=34, numberOfStores=37,
> numberOfStorefiles=52, storefileIndexSizeMB=1, rootIndexSizeKB=1084,
> totalStaticIndexSizeKB=29868, totalStaticBloomSizeKB=0, memstoreSizeMB=919,
> mbInMemoryWithoutWAL=286, numberOfPutsWithoutWAL=3532813,
> readRequestsCount=1299409247, writeRequestsCount=15476790,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1575, maxHeapMB=2933,
> blockCacheSizeMB=502.99, blockCacheFreeMB=230.39, blockCacheCount=7863,
> blockCacheHitCount=4998976142, blockCacheMissCount=4223771,
> blockCacheEvictedCount=3568722, blockCacheHitRatio=99%,
> blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=97,
> slowHLogAppendCount=0, fsReadLatencyHistogramMean=1488798.95,
> fsReadLatencyHistogramCount=2956662, fsReadLatencyHistogramMedian=239500,
> fsReadLatencyHistogram75th=287750, fsReadLatencyHistogram95th=654900,
> fsReadLatencyHistogram99th=1532570, fsReadLatencyHistogram999th=26074061,
> fsPreadLatencyHistogramMean=3522484.19,
> fsPreadLatencyHistogramCount=1264015,
> fsPreadLatencyHistogramMedian=1426500, fsPreadLatencyHistogram75th=1778250,
> fsPreadLatencyHistogram95th=15080950, fsPreadLatencyHistogram99th=25166530,
> fsPreadLatencyHistogram999th=114129728,
> fsWriteLatencyHistogramMean=1253190.67,
> fsWriteLatencyHistogramCount=747208, fsWriteLatencyHistogramMedian=409500,
> fsWriteLatencyHistogram75th=653500, fsWriteLatencyHistogram95th=2717950,
> fsWriteLatencyHistogram99th=4138890, fsWriteLatencyHistogram999th=41532109
> 2012-11-09 01:43:53,398 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled
> exception: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT
> rejected; currently processing slave181,60020,1352287478815 as dead server
> 2012-11-09 01:43:53,398 INFO org.apache.hadoop.ipc.HBaseServer: Stopping
> server on 60020
> ....
> 2012-11-09 01:43:53,403 INFO org.apache.hadoop.ipc.HBaseServer: Stopping
> IPC Server listener on 60020
> 2012-11-09 01:43:53,405 INFO
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: Sending interrupt to
> stop the worker thread
> 2012-11-09 01:43:53,405 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
> 2012-11-09 01:43:53,405 INFO org.apache.hadoop.ipc.HBaseServer: Stopping
> IPC Server Responder
> 2012-11-09 01:43:53,415 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 30 on 60020: exiting
> ...
> 2012-11-09 01:43:53,405 INFO
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: SplitLogWorker
> interrupted while waiting for task, exiting: java.lang.InterruptedException
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 38 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 40 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 39 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: SplitLogWorker
> slave181,60020,1352287478815 exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 41 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 37 on 60020: exiting
> 2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 34 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 47 on 60020: exiting
> 2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 33 on 60020: exiting
> 2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 18 on 60020: exiting
> 2012-11-09 01:43:53,418 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 32 on 60020: exiting
> 2012-11-09 01:43:53,415 INFO org.apache.hadoop.ipc.HBaseServer: Stopping
> IPC Server Responder
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 49 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 48 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 46 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 45 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 43 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 36 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 44 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 31 on 60020: exiting
> 2012-11-09 01:43:53,419 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 42 on 60020: exiting
> 2012-11-09 01:43:53,539 INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher:
> regionserver60020.cacheFlusher exiting
> 2012-11-09 01:43:53,539 INFO
> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
> 2012-11-09 01:43:53,540 DEBUG
> org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
> started; Attempting to free -48043.02 KB of total=503.11 MB
> 2012-11-09 01:43:53,540 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionChecker:
> regionserver60020.compactionChecker exiting
> 2012-11-09 01:43:53,540 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02
> 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
> 2012-11-09 01:43:53,540 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02
> 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.: disabling
> compactions & flushes
> 2012-11-09 01:43:53,541 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02
> 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
> 2012-11-09 01:43:53,541 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
> 2012-11-09 01:43:53,541 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,541 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
> 2012-11-09 01:43:53,542 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: aborting server
> slave181,60020,1352287478815
> 2012-11-09 01:43:53,542 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@66447828
> 2012-11-09 01:43:53,543 INFO
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> Closed zookeeper sessionid=0x13ada9d738a000b
> 2012-11-09 01:43:53,543 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97
> \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
> 2012-11-09 01:43:53,543 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97
> \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,543 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97
> \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
> 2012-11-09 01:43:53,544 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed record_info
> 2012-11-09 01:43:53,544 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02
> 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
> 2012-11-09 01:43:53,544 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_record_table,\xE5\xB9\xB8\xE8\xBF\x90\xE4\xB9\x8B\xE6\x98\x9F_\xE4\xBA\xA4\xE6\x98\x93\xE5\xA4\xA9\xE5\xA0\x82|13259799606|2012-11-02
> 15:42:26+1,1352143061469.70e80050c78686302a99a9d29d2de159.
> 2012-11-09 01:43:53,544 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
> 2012-11-09 01:43:53,544 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed search_info
> 2012-11-09 01:43:53,544 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,545 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
> 2012-11-09 01:43:53,545 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
> 2012-11-09 01:43:53,545 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_search_keyword_table,20121102|\xE8\xB2\x82\xE7\xBB\x92\xE8\xA1\xAB\xE5\xA5\xB3\xE5\xBC\x8F\xE7\xB2\xBE\xE5\x93\x81|1_38,1351877355995.d6edeead141dfa543e9e2d3e19080619.
> 2012-11-09 01:43:53,545 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
> 2012-11-09 01:43:53,545 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,545 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
> 2012-11-09 01:43:53,546 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed search_info
> 2012-11-09 01:43:53,546 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97
> \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
> 2012-11-09 01:43:53,546 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_search_keyword_cat_table,20121102|\xE5\xA5\xB3\xE8\xA5\xBF\xE8\xA3\x85\xE5\xA4\x96\xE5\xA5\x97
> \xE7\xA7\x8B|50008090,1351862890985.b0815ad291a4f7ba51c8f493eedf9290.
> 2012-11-09 01:43:53,546 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16
> 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
> 2012-11-09 01:43:53,547 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16
> 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.: disabling
> compactions & flushes
> 2012-11-09 01:43:53,547 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16
> 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
> 2012-11-09 01:43:53,547 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed redirect_info
> 2012-11-09 01:43:53,548 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed redirect_info
> 2012-11-09 01:43:53,548 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
> 2012-11-09 01:43:53,548 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_redirect_table,20121101|\xE7\x93\x9C\xE7\x89\x87|9,1351729853756.37637cdfb34c03cb082bd3c580182e94.
> 2012-11-09 01:43:53,548 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23
> 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
> 2012-11-09 01:43:53,548 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
> 2012-11-09 01:43:53,548 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_redirect_table,20121104|\xE8\x84\x9A\xE9\x83\xA8\xE6\x8C\x89\xE6\x91\xA9\xE5\x99\xA8\xE6\x9C\xA8\xE8\xB4\xA8|11,1352109179634.e2d22bc249913ce71fb15e7a63ecaf9f.
> 2012-11-09 01:43:53,548 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23
> 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.: disabling
> compactions & flushes
> 2012-11-09 01:43:53,548 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23
> 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
> 2012-11-09 01:43:53,548 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26
> 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
> 2012-11-09 01:43:53,549 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26
> 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.: disabling
> compactions & flushes
> 2012-11-09 01:43:53,549 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26
> 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
> 2012-11-09 01:43:53,549 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed record_info
> 2012-11-09 01:43:53,550 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16
> 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
> 2012-11-09 01:43:53,550 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_record_table,osa\xE5\x93\x81\xE7\x89\x8C\xE6\x9C\x8D\xE9\xA5\xB0\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|16232863706|2012-10-16
> 10:04:16+1,1351741141541.2a5e974c8911051eb51d99a18100fa0c.
> 2012-11-09 01:43:53,550 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
> 2012-11-09 01:43:53,550 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,550 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
> 2012-11-09 01:43:53,555 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed record_info
> 2012-11-09 01:43:53,555 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26
> 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
> 2012-11-09 01:43:53,555 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_record_table,\xE5\xBE\xA1\xE6\xB3\xA5\xE5\x9D\x8A\xE6\x97\x97\xE8\x88\xB0\xE5\xBA\x97|2224483487|2012-08-26
> 22:36:57+1,1352359403922.91c27c0735ebb0402c07f5f2314b9f8e.
> 2012-11-09 01:43:53,555 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
> 2012-11-09 01:43:53,555 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,555 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
> 2012-11-09 01:43:53,557 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed redirect_info
> 2012-11-09 01:43:53,557 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_redirect_table,20121101|\xE6\xB1\xBD\xE6\xB2\xB9\xE6\x89\x93\xE7\x81\xAB\xE6\x9C\xBA|2,1351729853756.460333f7b27a7ea4bffe37693d4530a7.
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> 2012-11-09 01:43:53,558 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed redirect_info
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> 2012-11-09 01:43:53,558 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_redirect_table,20121107|\xE8\xAF\xBA\xE7\xBE\x8E\xE4\xB8\xBD\xE8\x8E\x8E\xE5\xB7\xA6\xE6\x97\x8B\xE7\xBB\xB4c\xE9\x9D\xA2\xE8\x86\x9C|22,1352310337700.90298668add8aaca607f17f7102cf1df.
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28
> 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28
> 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.: disabling
> compactions & flushes
> 2012-11-09 01:43:53,558 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28
> 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
> 2012-11-09 01:43:53,559 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed record_info
> 2012-11-09 01:43:53,559 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23
> 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
> 2012-11-09 01:43:53,560 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE6\x97\xB6\xE5\xB0\x9A\xE9\x83\xA8\xE8\x90\xBD|20321856048|2012-10-23
> 21:16:28+1,1351876273299.d427de42c14b7faff9e1dd67fd2fcb6b.
> 2012-11-09 01:43:53,560 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
> 2012-11-09 01:43:53,560 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,560 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
> 2012-11-09 01:43:53,566 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed item_info
> 2012-11-09 01:43:53,566 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> 2012-11-09 01:43:53,566 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_item_redirect_table,14428874253,1351721298220.18d2c63be7451b8e12b368552910bb20.
> 2012-11-09 01:43:53,566 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
> 2012-11-09 01:43:53,566 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,566 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
> 2012-11-09 01:43:53,568 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed record_info
> 2012-11-09 01:43:53,569 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28
> 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
> 2012-11-09 01:43:53,569 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_record_table,\xE5\xB0\x8F\xE7\x8B\x9712921|8276006227|2012-10-28
> 19:32:40+1,1352356446256.a2c9b8d302eaa8d077a791e30797f393.
> 2012-11-09 01:43:53,569 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
> 2012-11-09 01:43:53,569 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,569 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
> 2012-11-09 01:43:53,570 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed doc_info
> 2012-11-09 01:43:53,570 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed doc_pic
> 2012-11-09 01:43:53,573 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed item_info
> 2012-11-09 01:43:53,574 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
> 2012-11-09 01:43:53,574 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_item_table,tearing_angel|7924817804,1352010672212.38d792b0b7a10c34f1119e50a1d87e61.
> 2012-11-09 01:43:53,574 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> 2012-11-09 01:43:53,574 INFO org.apache.zookeeper.ZooKeeper: Session:
> 0x13ada9d738a000b closed
> 2012-11-09 01:43:53,574 INFO org.apache.zookeeper.ClientCnxn: EventThread
> shut down
> 2012-11-09 01:43:53,574 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,574 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on 21 regions
> to close
> 2012-11-09 01:43:53,574 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> 2012-11-09 01:43:53,575 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed doc_stat
> 2012-11-09 01:43:53,575 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed reply_data
> 2012-11-09 01:43:53,576 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
> 2012-11-09 01:43:53,576 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> yq_source_document,499A080BB3BEF93B9C6FF24F4103E7554,1352217737193.5b2ca166dc37841bdecb51e4cb942914.
> 2012-11-09 01:43:53,576 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
> 2012-11-09 01:43:53,576 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,576 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
> 2012-11-09 01:43:53,578 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed search_info
> 2012-11-09 01:43:53,578 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
> 2012-11-09 01:43:53,579 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_search_keyword_table,20121106|\xE4\xB8\x93\xE4\xB8\x9A\xE5\x8A\x9F\xE6\x94\xBE1200w|1_5,1352226330801.ea59db2af5de9398565b94d54092766f.
> 2012-11-09 01:43:53,579 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
> 2012-11-09 01:43:53,579 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,579 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
> 2012-11-09 01:43:53,590 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed item_info
> 2012-11-09 01:43:53,590 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> 2012-11-09 01:43:53,590 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_item_redirect_table,15762041897,1352117930928.9249f776d3aa6a903869dbeb6aeef49f.
> 2012-11-09 01:43:53,590 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
> 2012-11-09 01:43:53,591 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,591 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
> 2012-11-09 01:43:53,593 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed img_info
> 2012-11-09 01:43:53,593 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
> 2012-11-09 01:43:53,593 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> yq_source_img,445B8F84DB239AD68C890F631D9AD654D1D05D9D4828331B72CA9165ACC54AEB72,1352297685797.08872b99b456b1f7e80c84bc74f3d067.
> 2012-11-09 01:43:53,593 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
> 2012-11-09 01:43:53,593 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,593 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
> 2012-11-09 01:43:53,597 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed item_info
> 2012-11-09 01:43:53,597 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
> 2012-11-09 01:43:53,597 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_item_table,\xE4\xB8\x83\xE5\x8C\xB9\xE7\x8B\xBC\xE7\xAB\x8B\xE6\xB7\x98\xE4\xB8\x93\xE5\x8D\x96\xE5\xBA\x97|14047700550,1352010672212.f59c5198deea65c4bcd642919ca7ae42.
> 2012-11-09 01:43:53,597 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
> 2012-11-09 01:43:53,597 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,597 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
> 2012-11-09 01:43:53,598 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed info
> 2012-11-09 01:43:53,598 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
> 2012-11-09 01:43:53,598 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_redirect_shop_item_table,\xE6\x95\x8F\xE7\x91\x9E\xE6\xB1\xBD\xE8\xBD\xA6\xE7\x94\xA8\xE5\x93\x81\xE6\x89\xB9\xE5\x8F\x91\xE5\x95\x86\xE8\xA1\x8C|12644774189|20121101,1351873340363.1b4972a0bc71f9ab9826c545bdedae17.
> 2012-11-09 01:43:53,598 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
> 2012-11-09 01:43:53,598 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.: disabling
> compactions & flushes
> 2012-11-09 01:43:53,598 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
> 2012-11-09 01:43:53,599 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed img_info
> 2012-11-09 01:43:53,599 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
> 2012-11-09 01:43:53,599 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> yq_source_img,44772533102745183A2408224FEC505DC2DC92D3B3E09A9B5FA5BC9DC4BA80CB0B,1352297685797.1a964cf1be0d4b0aa5829ba2d468aeb0.
> 2012-11-09 01:43:53,600 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
> 2012-11-09 01:43:53,600 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,600 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
> 2012-11-09 01:43:53,603 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed info
> 2012-11-09 01:43:53,603 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
> 2012-11-09 01:43:53,603 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_redirect_shop_item_table,\xE5\xA4\xA9\xE5\xB5\xA9\xE9\xA3\x9F\xE5\x93\x81\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|12620601137|20121101,1351873340363.530333489d35966d4472a051b51fb6a8.
> 2012-11-09 01:43:53,603 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15
> 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
> 2012-11-09 01:43:53,603 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15
> 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.: disabling
> compactions & flushes
> 2012-11-09 01:43:53,603 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15
> 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
> 2012-11-09 01:43:53,605 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed item_info
> 2012-11-09 01:43:53,606 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_item_his_table,,1352369543266.2bebca6052be10e59932329bcaf18879.
> 2012-11-09 01:43:53,606 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed record_info
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of taobao_search_keyword_table,20121103|\xE7\x94\xB7\xE5\xA3\xAB
> \xE8\x83\x8C\xE5\xBF\x83
> \xE5\xAE\xBD\xE6\x9D\xBE|1_24,1351960783892.5902777356e5a6994b0e267991957df0.
> 2012-11-09 01:43:53,606 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15
> 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_search_keyword_table,20121103|\xE7\x94\xB7\xE5\xA3\xAB
> \xE8\x83\x8C\xE5\xBF\x83
> \xE5\xAE\xBD\xE6\x9D\xBE|1_24,1351960783892.5902777356e5a6994b0e267991957df0.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_record_table,\xE6\x88\x91\xE7\x9A\x84\xE5\xAE\x9D\xE8\xB4\x9D\xE5\xB0\xB1\xE6\x98\xAF\xE5\xA5\xBD|1948882565|2012-08-15
> 23:39:04+1,1352359403922.e63368f3d154631bf3ea40f12cc8d4f1.
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_search_keyword_table,20121103|\xE7\x94\xB7\xE5\xA3\xAB
> \xE8\x83\x8C\xE5\xBF\x83
> \xE5\xAE\xBD\xE6\x9D\xBE|1_24,1351960783892.5902777356e5a6994b0e267991957df0.
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_table,20121103|\xE9\x99\xB6\xE7\x93\xB7\xE6\xB0\xB4\xE6\xA7\xBD\xE6\xB4\x97\xE8\xA1\xA3\xE7\x9B\x86|110,1352020981378.50241b8ed071b624f785064cac238367.
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_table,20121103|\xE9\x99\xB6\xE7\x93\xB7\xE6\xB0\xB4\xE6\xA7\xBD\xE6\xB4\x97\xE8\xA1\xA3\xE7\x9B\x86|110,1352020981378.50241b8ed071b624f785064cac238367.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,606 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_table,20121103|\xE9\x99\xB6\xE7\x93\xB7\xE6\xB0\xB4\xE6\xA7\xBD\xE6\xB4\x97\xE8\xA1\xA3\xE7\x9B\x86|110,1352020981378.50241b8ed071b624f785064cac238367.
> 2012-11-09 01:43:53,607 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed record_info
> 2012-11-09 01:43:53,607 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
> 2012-11-09 01:43:53,607 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_record_table,\xE5\x98\x89\xE5\xA3\xAB\xE5\x86\x85\xE8\xA1\xA3\xE4\xB8\x93\xE8\x90\xA5\xE5\xBA\x97|17553619677|2012-11-01
> 09:52:35+1,1352356446256.77c58ab3b4586478b84f4922950d08e6.
> 2012-11-09 01:43:53,607 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
> 2012-11-09 01:43:53,608 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,608 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
> 2012-11-09 01:43:53,609 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed top_info
> 2012-11-09 01:43:53,610 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
> 2012-11-09 01:43:53,610 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_top_keyword_table,20121101|50011744|\xE6\x90\x9C\xE7\xB4\xA2\xE4\xB8\x8A\xE5\x8D\x87|181,1351801768479.660bb44aacacc6cdfebc62b1ea7c0aae.
> 2012-11-09 01:43:53,610 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
> 2012-11-09 01:43:53,610 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,610 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
> 2012-11-09 01:43:53,613 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed item_info
> 2012-11-09 01:43:53,613 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
> 2012-11-09 01:43:53,613 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_item_his_table,ji0101|3866728732|20121107,1352369543266.79142c52ab2c3b1eb7bd975b8191960f.
> 2012-11-09 01:43:53,614 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
> 2012-11-09 01:43:53,614 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,614 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
> 2012-11-09 01:43:53,616 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed redirect_info
> 2012-11-09 01:43:53,616 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Closed
> taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
> 2012-11-09 01:43:53,616 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_redirect_table,20121102|\xE5\xB7\xA7\xE8\x99\x8E\xE6\xAD\xA3\xE7\x89\x88|21,1351883700568.0314d7b352cbde974d5dabffbbfbd2b9.
> 2012-11-09 01:43:53,616 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Processing
> close of
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
> 2012-11-09 01:43:53,616 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Closing
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.:
> disabling compactions & flushes
> 2012-11-09 01:43:53,616 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region
> taobao_redirect_table,20121107|\xE9\xA9\xBE\xE9\xA9\xB6\xE8\xAF\x81\xE5\xA5\x97
> \xE6\x83\x85\xE4\xBE\xA3|4,1352378961446.9502c1830ad553d1e85497f9445d30d3.
> 2012-11-09 01:43:53,619 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> closed redirect_info
> ....
> 2012-11-09 01:43:53,625 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler: Closed
> region
> taobao_item_his_table,tearing_angel|10422969928|20121101,1351793039530.278ef97228af5cf2ccf547ef332450d3.
> 2012-11-09 01:43:53,696 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server master/192.168.2.167:2222
> 2012-11-09 01:43:53,696 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-11-09 01:43:53,696 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to master/192.168.2.167:2222, initiating session
> 2012-11-09 01:43:53,697 INFO org.apache.zookeeper.ClientCnxn: Unable to
> reconnect to ZooKeeper service, session 0x13ada9d738a0001 has expired,
> closing socket connection
> 2012-11-09 01:43:53,697 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> slave181,60020,1352287478815: regionserver:60020-0x13ada9d738a0001
> regionserver:60020-0x13ada9d738a0001 received expired from ZooKeeper,
> aborting
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:389)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:286)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
> 2012-11-09 01:43:53,700 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort:
> loaded coprocessors are: []
> 2012-11-09 01:43:53,702 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=34, numberOfStores=37,
> numberOfStorefiles=52, storefileIndexSizeMB=1, rootIndexSizeKB=1084,
> totalStaticIndexSizeKB=29868, totalStaticBloomSizeKB=0, memstoreSizeMB=919,
> mbInMemoryWithoutWAL=286, numberOfPutsWithoutWAL=3532813,
> readRequestsCount=1299409247, writeRequestsCount=15476790,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1601, maxHeapMB=2933,
> blockCacheSizeMB=502.99, blockCacheFreeMB=230.39, blockCacheCount=7863,
> blockCacheHitCount=4998976142, blockCacheMissCount=4223771,
> blockCacheEvictedCount=3568722, blockCacheHitRatio=99%,
> blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=97,
> slowHLogAppendCount=0, fsReadLatencyHistogramMean=1488798.95,
> fsReadLatencyHistogramCount=2956662, fsReadLatencyHistogramMedian=239500,
> fsReadLatencyHistogram75th=287750, fsReadLatencyHistogram95th=654900,
> fsReadLatencyHistogram99th=1532570, fsReadLatencyHistogram999th=26074061,
> fsPreadLatencyHistogramMean=3522484.19,
> fsPreadLatencyHistogramCount=1264015,
> fsPreadLatencyHistogramMedian=1426500, fsPreadLatencyHistogram75th=1778250,
> fsPreadLatencyHistogram95th=15080950, fsPreadLatencyHistogram99th=25166530,
> fsPreadLatencyHistogram999th=114129728,
> fsWriteLatencyHistogramMean=1253190.67,
> fsWriteLatencyHistogramCount=747208, fsWriteLatencyHistogramMedian=409500,
> fsWriteLatencyHistogram75th=653500, fsWriteLatencyHistogram95th=2717950,
> fsWriteLatencyHistogram99th=4138890, fsWriteLatencyHistogram999th=41532109
> 2012-11-09 01:43:53,708 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> regionserver:60020-0x13ada9d738a0001 regionserver:60020-0x13ada9d738a0001
> received expired from ZooKeeper, aborting
> 2012-11-09 01:43:53,708 INFO org.apache.zookeeper.ClientCnxn: EventThread
> shut down
> 2012-11-09 01:43:53,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server
> slave181,60020,1352287478815; all regions closed.
> 2012-11-09 01:43:53,778 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: regionserver60020.logSyncer
> exiting
> 2012-11-09 01:43:53,781 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.HLog: closing hlog writer in
> hdfs://master:54310/hbase/.logs/slave181,60020,1352287478815
> 2012-11-09 01:43:53,839 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Close and delete failed
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException:
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on
> /hbase/.logs/slave181,60020,1352287478815/slave181%2C60020%2C1352287478815.1352395531200
> File does not exist. [Lease. Holder:
> DFSClient_hb_rs_slave181,60020,1352287478815, pendingcreates: 1]
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1631)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1622)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:1677)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:1665)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:718)
> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
> at
> org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeWAL(HRegionServer.java:945)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:800)
> at java.lang.Thread.run(Thread.java:722)
> 2012-11-09 01:43:53,840 INFO org.apache.hadoop.hbase.regionserver.Leases:
> regionserver60020 closing leases
> 2012-11-09 01:43:53,840 INFO org.apache.hadoop.hbase.regionserver.Leases:
> regionserver60020 closed leases
> 2012-11-09 01:43:53,840 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting for Split
> Thread to finish...
> 2012-11-09 01:43:53,840 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting for Large
> Compaction Thread to finish...
> 2012-11-09 01:43:53,840 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Waiting for Small
> Compaction Thread to finish...
> 2012-11-09 01:43:53,850 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
> 2012-11-09 01:43:53,850 INFO org.apache.hadoop.hbase.util.RetryCounter:
> Sleeping 2000ms before retry #1...
> 2012-11-09 01:43:55,478 INFO org.apache.hadoop.hbase.regionserver.Leases:
> regionserver60020.leaseChecker closing leases
> 2012-11-09 01:43:55,481 INFO org.apache.hadoop.hbase.regionserver.Leases:
> regionserver60020.leaseChecker closed leases
> 2012-11-09 01:43:55,852 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
> 2012-11-09 01:43:55,852 INFO org.apache.hadoop.hbase.util.RetryCounter:
> Sleeping 4000ms before retry #2...
> 2012-11-09 01:43:59,854 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
> 2012-11-09 01:43:59,854 INFO org.apache.hadoop.hbase.util.RetryCounter:
> Sleeping 8000ms before retry #3...
> 2012-11-09 01:44:07,856 WARN
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
> 2012-11-09 01:44:07,856 ERROR
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper delete
> failed after 3 retries
> 2012-11-09 01:44:07,856 WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Failed deleting my
> ephemeral node
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /hbase/rs/slave181,60020,1352287478815
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:868)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:127)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:974)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:963)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1037)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:815)
> at java.lang.Thread.run(Thread.java:722)
> 2012-11-09 01:44:07,857 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server
> slave181,60020,1352287478815; zookeeper connection closed.
> 2012-11-09 01:44:07,857 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver60020
> exiting
> 2012-11-09 01:44:07,875 INFO
> org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook starting;
> hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-5,5,main]
> 2012-11-09 01:44:07,875 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Shutdown hook
> 2012-11-09 01:44:07,875 INFO
> org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs shutdown
> hook thread.
> 2012-11-09 01:44:07,876 INFO
> org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook finished
>
> LOG
> END--------------------------------------------------------------------------------------------------------
>
>
>
> --------------
>
> 兰锋成
> Tel: 0592-5900539/Fax: 0592-5900395
> Mobel: 13599927214
> Email: lanfengcheng@xmtongrong.com
> Website: www.xmtongrong.com
> 厦门瑞景商业广场 1#6 层 1-601 单元
> Room601, building No.1 Ruijing Shopping Mall, Siming District, Amoy