You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by hua beatls <be...@gmail.com> on 2013/06/15 17:00:03 UTC
WHY REGIONS STOP SPLITTING
HI,
we use CDH 4.2.0, find that some of the regions stop spliting and
increase contiusuly under heavy load. some regions become much bigger than
the setting value(4G),reaches 46g.
any reason or bugs?
Thansk!
beatls
Re: WHY REGIONS STOP SPLITTING
Posted by Ted Yu <yu...@gmail.com>.
org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handled SPLIT
event;
parent=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371276011390.10fc985bae394d3de78d2c5595a7ee4b.
daughter
a=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371277127973.060e12898838607dd55d6bb0de074af6.daughter
b=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5g>\xEC,1371277127973.0584b8152c974c7b22a618a9561f303e.
Was 1371276011390.10fc985bae394d3de78d2c5595a7ee4b. the region that grew to
46GB ?
The above log didn't seem to match region server log you posted later - I
couldn't find 1371276011390.10fc985bae394d3de78d2c5595a7ee4b. in region
server log.
On Sat, Jun 15, 2013 at 8:18 AM, hua beatls <be...@gmail.com> wrote:
> hi, TED,
> below is part of log from master server:
> 2013-06-15 14:18:18,262 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> 2013-06-15 14:18:26,147 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:26,153 INFO org.apache.zookeeper.ZooKeeper: Initiating
> client connection, connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> sessionTimeout=180000
>
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:18:26,155 INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> this process is 72221@dsjhd001
> 2013-06-15 14:18:26,155 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> 2013-06-15 14:18:26,155 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server dsjhd003/132.228.248.13:2181. Will not attempt
> to authenticate using SASL (unknown error)
> 2013-06-15 14:18:26,157 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to dsjhd003/132.228.248.13:2181, initiating session
> 2013-06-15 14:18:26,168 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server dsjhd003/132.228.248.13:2181, sessionid =
> 0x33f38cfddfd0598, negotiated timeout = 40000
> 2013-06-15 14:18:26,172 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row= for max=2147483647 rows using
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:18:26,210 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Creating scanner over .META. starting at key 'receipt,,'
> 2013-06-15 14:18:26,210 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Advancing internal scanner to startKey at 'receipt,,'
> 2013-06-15 14:18:26,213 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Creating scanner over .META. starting at key 'receipt,,'
> 2013-06-15 14:18:26,213 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Advancing internal scanner to startKey at 'receipt,,'
> 2013-06-15 14:18:26,473 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ZooKeeper: Session:
> 0x33f38cfddfd0598 closed
> 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ClientCnxn: EventThread
> shut down
> 2013-06-15 14:18:26,485 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row=receipt,,00000000000000 for max=2147483647
> rows using
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:18:26,491 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001110007251002,1371260257669.f0f8b131fb77efb81f16eb01e493953f.
> 2013-06-15 14:18:26,492 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001110007251002,1371261674696.a9d0635ea62ea83eb847e2dc49c1c3a3.
> 2013-06-15 14:18:26,493 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001111085351002,1371261674696.5f600129659614930a10215e81b551d7.
> 2013-06-15 14:18:26,494 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001112163491002,1371260257669.c0358d1d04b4080d32199262c835a1fa.
> 2013-06-15 14:18:26,495 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001130002641002,1371260606344.c6eb5c6d48c5a2b2ae7612a287273108.
> 2013-06-15 14:18:26,496 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001131034981002,1371260606344.221332a360f91bb525a39dfdbe47a8b3.
> 2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001132067341002,1371260431691.5469992ccbb963ef7d27f53c2e05c88d.
> 2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001141017741002,1371260431691.61fada90a01436cce4342baae8b1a341.
> 2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001190035581002,1371178883452.cf8f2ad83efefa816340cd920bc3bbf8.
> 2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001210077421002,1371178883452.d8de7e1c6e7568c76a5337e3c53162fc.
> 2013-06-15 14:18:26,500 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001210077421002,1371211272356.2b646d6d9433dc78315756d491ac8e63.
> 2013-06-15 14:18:26,501 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001210077421002,1371213862480.3c695d05eaf2342e015149947dfccce2.
> 2013-06-15 14:18:26,502 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001210077421002,1371215646288.bc0ae5fea3f27f66e87a6e6960e6ea22.
> 2013-06-15 14:18:26,503 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001210854261002,1371215646288.c754faa28ce47cb12c72ecbbc1f7c85d.
> 2013-06-15 14:18:26,504 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001211631141002,1371213862480.9ac7561a2297b5723c9bde2ac82a275b.
> 2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001211631141002,1371215537292.c86caaeca975e84508593176d2403726.
> 2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001220360631002,1371215537292.97757f6b0418dea7294af03a3fedfac1.
> 2013-06-15 14:18:26,506 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001221137531002,1371211272356.79edc478ea385978cd954025dac8b9f9.
> 2013-06-15 14:18:26,507 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001221137531002,1371213598407.9b1bbd5cd11099c5fedfd95bab0915df.
> 2013-06-15 14:18:26,508 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001221137531002,1371215427123.357463550f00d07ffb25df0d84e0b837.
> 2013-06-15 14:18:26,509 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001221914391002,1371215427123.71d31890d25ad0b6677a705cc7d71c37.
> 2013-06-15 14:18:26,510 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001230648541002,1371213598407.e6bb294918d09dc08fd309807e208a2a.
> 2013-06-15 14:18:26,511 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001230648541002,1371215306956.0c4e3c9a1cedcd40cca686a8b70bd980.
> 2013-06-15 14:18:26,512 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001231425401002,1371215306956.283bada01f51766d0280219822c99a69.
> 2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001280031821002,1371178911107.c0f3a84ffb0c39ed368941f6671d0cfd.
> 2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001300080491002,1371178911107.27b811538f6ee109f267653f5f560b87.
> 2013-06-15 14:18:26,515 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001330007771002,1371181427502.f6b2040f8a16b60107d71e32a810a371.
> 2013-06-15 14:18:26,516 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001330007771002,1371207552053.add80b8254579a34fbbf070c0bb605bd.
> 2013-06-15 14:18:26,517 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001331064311002,1371207552053.fe632f34b69579eccbd8b4e813b9088d.
> 2013-06-15 14:18:26,518 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001340042061002,1371181427502.747aa41330a84f7d5fe696defc10f985.
> 2013-06-15 14:18:26,519 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001380041871002,1371180369431.e390609ca6beb9b7baf8f2b4796f9dda.
> 2013-06-15 14:18:26,520 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001390138251002,1371180369431.23d67b78df8fb952861b6d60073b643a.
> 2013-06-15 14:18:26,521 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001390138251002,1371210229806.6f0cd39d615be35106aeec04c526fcda.
> 2013-06-15 14:18:26,522 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001390138251002,1371212995795.4f9df026d5b9026f4282edcc4f54a1ef.
> 2013-06-15 14:18:26,523 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001391127371002,1371212995795.a62a6b8d13925f98ce7b9243b19b3d25.
> 2013-06-15 14:18:26,524 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001400098641002,1371210229806.55941435b45764299ec5b6563ecf5d4b.
> 2013-06-15 14:18:26,525 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001400098641002,1371212823051.ce6a942475c3223c26544beedd04594d.
> 2013-06-15 14:18:26,526 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001401087781002,1371212823051.73837e8ba4a0598fecf560cb4a14b5e4.
> 2013-06-15 14:18:26,527 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001410067721002,1371259508101.57037bf83841725172996ef4d9c1a7ee.
> 2013-06-15 14:18:26,528 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001420069141002,1371259508101.9c1caf332f6325b60a087467c2b48fa9.
> 2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001420069141002,1371261843284.937d2e422a6130b82228443ac68afd02.
> 2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001421136541002,1371261843284.961e6d73f48f108fb82bcacbeb488775.
> 2013-06-15 14:18:26,530 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001430035471002,1371258634059.1b8e1a437453a9e4529179205c5ba625.
> 2013-06-15 14:18:26,531 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001440056171002,1371258634059.4e2ec5bdd3a8fcd599b1edaa4243333f.
> 2013-06-15 14:18:26,532 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001440056171002,1371262374869.0acf2c312d53e933d55d36d6e6558abc.
> 2013-06-15 14:18:26,533 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001441098211002,1371262374869.15bc8b334ff9407679903be126d0805d.
> 2013-06-15 14:18:26,537 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001490005171002,1371180009315.660270fd8baafce2f0607a2996fb81e9.
> 2013-06-15 14:18:26,538 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001490005171002,1371207839796.804b59d6e59d8d9c2506057cafa1b50f.
> 2013-06-15 14:18:26,539 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001490005171002,1371209785865.96ec8a1b777cf3e93eb5f7e173455698.
> 2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001490736191002,1371209785865.caf571b498f5d747c9dc77fcd290c161.
> 2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001491467231002,1371207839796.74ca8bcf079e8281a8d5403e5ac9b1f3.
> 2013-06-15 14:18:26,541 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001491467231002,1371209683706.b10e30402bf0637b915947c640fd7ca4.
> 2013-06-15 14:18:26,542 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001500716371002,1371209683706.899d40e11dc4e5930a4ad49ad14d831b.
> 2013-06-15 14:18:26,543 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001510010721002,1371180009315.9fd04413854e814904405e6298ea4ba4.
> 2013-06-15 14:18:26,545 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001600069801002,1371181237169.f21a35fac1906ace208dda777b010514.
> 2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001600069801002,1371208329446.5e56a0c7fc4ccc83d56b09abd586794c.
> 2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001600869361002,1371208329446.47ead382d396a937c1c25a18da6d7138.
> 2013-06-15 14:18:26,547 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001610159121002,1371181237169.186c47cc02f5258d4f86699ece9b7823.
> 2013-06-15 14:18:26,549 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001870007081002,1371259179922.8d20b1a06529454ec2d7f1f4247bf22e.
> 2013-06-15 14:18:26,550 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001890716531002,1371259179922.4b096c2c516438baa3668ce235fc9c90.
> 2013-06-15 14:18:26,551 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001890716531002,1371262229170.d40579d02a66226656a1e48e0aa5fef0.
> 2013-06-15 14:18:26,552 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001890716531002,1371263273244.72b3118398132ecf0b630a3aeaa0c588.
> 2013-06-15 14:18:26,553 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001900211511002,1371263273244.62bd89081bfbc0e1bfca7db214eebc8c.
> 2013-06-15 14:18:26,554 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001901130951002,1371262229170.658e697026a46c7a1097213c2b7dbc59.
> 2013-06-15 14:18:26,555 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001901130951002,1371263138497.f9e6520a49c52462e1a888bc3e62f71a.
> 2013-06-15 14:18:26,556 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001910587571002,1371263138497.3077fae3eeafdb11ae663bd7c6761418.
> 2013-06-15 14:18:26,557 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001920003561002,1371177634606.6805f9fffb6384fc6befe8f4ef2c8478.
> 2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001960038371002,1371177634606.61662ddd932e0e483adac0c0ad072d31.
> 2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001960038371002,1371179042477.7ea6904a4aca54b84f006dcffa98353e.
> 2013-06-15 14:18:26,559 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001960038371002,1371200342086.1f6ce38a8765a5de4dced1f616de590b.
> 2013-06-15 14:18:26,560 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001960038371002,1371202134689.f6385d90b1d6c3dd0fad4a5207a632cc.
> 2013-06-15 14:18:26,561 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001960038371002,1371203337958.f80a9a7182b7346c16b0e50565ead084.
> 2013-06-15 14:18:26,562 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001960966991002,1371203337958.0969455703fbd4594fba90eca1a5b8a7.
> 2013-06-15 14:18:26,563 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001961895651002,1371202134689.709bdc87c3f5cb60a0379bb55d9dc3f1.
> 2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001961895651002,1371203142268.293fc7628be8c32235b68914562f0134.
> 2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001962824291002,1371203142268.e6af9210314c5a4ac5e6f5a793c1b2f3.
> 2013-06-15 14:18:26,565 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001970486031002,1371200342086.7ea869039a9d75c2d58a36bd1e3f43e2.
> 2013-06-15 14:18:26,566 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001970486031002,1371201529596.c84ee1352bd7042f01c225a07ec9fda5.
> 2013-06-15 14:18:26,567 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001970486031002,1371202941763.6c4f92536fb9eede8da7f9a01d431b80.
> 2013-06-15 14:18:26,568 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001971414651002,1371202941763.969715998f0266c37e14e3ef89d3ea92.
> 2013-06-15 14:18:26,569 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001972343311002,1371201529596.fcdc2b301fa6d7792dd3e5535e8cc58a.
> 2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001972343311002,1371202732228.d091d6d827acc608724b10c5f2b72010.
> 2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001973271951002,1371202732228.b7dc98ba29c04b199d9c839beadfb7e6.
> 2013-06-15 14:18:26,571 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001980074991002,1371179042477.e8dd68c0f884a491e63f477cf2319164.
> 2013-06-15 14:18:26,572 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001980074991002,1371209818804.b634bc05efa2d4b6eb98f85ad4fde93d.
> 2013-06-15 14:18:26,573 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001980074991002,1371212403387.705ad001a08ff629c21c8038a7e87363.
> 2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001980074991002,1371214663565.b13dd722151a2ea44a6aba5d7af1905b.
> 2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001980074991002,1371216342982.25b9c7134d2baf28f85b7e588c42fa1f.
> 2013-06-15 14:18:26,575 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001980690071002,1371216342982.8a3ece0302a7aad860a9ce98e418d664.
> 2013-06-15 14:18:26,576 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001981305191002,1371214663565.7e7375671360e812eb6e7adeff39b3a2.
> 2013-06-15 14:18:26,577 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001981305191002,1371216258668.69aa7ba14b3748dd78a035fe0fa73994.
> 2013-06-15 14:18:26,578 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001981920291002,1371216258668.a9e3bd75ed74cffefe41b2622a489cda.
> 2013-06-15 14:18:26,579 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001982535431002,1371212403387.3364f1a6b67a83de7f0e9b62064f6b48.
> 2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001982535431002,1371214478503.a8e263e0ba0190e55da471542efb3fc9.
> 2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001982535431002,1371216173963.a7a72feecebdd2c4d21be2808b4550a8.
> 2013-06-15 14:18:26,581 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001990432891002,1371216173963.bf8da05c0e99900af0fbb1fc2c9a5b5a.
> 2013-06-15 14:18:26,582 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001991048011002,1371214478503.ea2e8bc4b27b0a1b440586a197d67d7f.
> 2013-06-15 14:18:26,583 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001991048011002,1371216090115.cbe8280d8b6ebbbb5386f1c34b1ab98b.
> 2013-06-15 14:18:26,584 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001991663111002,1371216090115.bb548f5af5e0d9c2a34e456ff23472e1.
> 2013-06-15 14:18:26,585 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001992278251002,1371209818804.a0e860fe725cc86184d8525d134d4344.
> 2013-06-15 14:18:26,587 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001992278251002,1371211949705.6b119a297db81b2177909e51d833358e.
> 2013-06-15 14:18:26,588 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001992278251002,1371214293670.f091e9463441e08f39e84074ab58f3f0.
> 2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000001992278251002,1371215973835.1a1e1125671301df2b54f1a0527daa18.
> 2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000002240013061002,1371215973835.42d223d52e9ecad03dd10ebe9542551e.
> 2013-06-15 14:18:26,590 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000002620017131002,1371214293670.a462b626169d5286149303e76eb83f52.
> 2013-06-15 14:18:26,591 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000002620017131002,1371215898321.aa059217f2f2788f3a23bad75ff34e26.
> 2013-06-15 14:18:26,592 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000011000071171002,1371215898321.3a4c342e75751ec441a2d63c3f06a5f2.
> 2013-06-15 14:18:26,593 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000011000686571002,1371211949705.3668041afa8e2cce5e8abb80a8d2de54.
> 2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000011000686571002,1371214112540.9a285ce01761253f977dbed2def5b953.
> 2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000011000686571002,1371215818016.2d2a6518a7bd35f0615fc990acec5238.
> 2013-06-15 14:18:26,595 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000011001301951002,1371215818016.799bf01965ad79f97e128f6c80938809.
> 2013-06-15 14:18:26,596 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000011001917351002,1371214112540.921f4f2af20f783f938223c49065b28e.
> 2013-06-15 14:18:26,597 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000011001917351002,1371215735171.f8b8b79d2aa33182a60f16aafe449524.
> 2013-06-15 14:18:26,598 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> 2013-06-15 14:18:29,377 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:30,761 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:36,248 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:37,049 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:37,735 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:38,416 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:39,139 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:39,933 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:40,627 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:41,285 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:42,298 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:42,949 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:43,635 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:47,984 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_SPLITTING, server=dsjhd006,60020,1371197351220,
> region=10fc985bae394d3de78d2c5595a7ee4b
> 2013-06-15 14:18:48,220 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_SPLIT, server=dsjhd006,60020,1371197351220,
> region=10fc985bae394d3de78d2c5595a7ee4b
> 2013-06-15 14:18:48,221 DEBUG
> org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handling SPLIT
> event for 10fc985bae394d3de78d2c5595a7ee4b; deleting node
> 2013-06-15 14:18:48,221 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:60000-0x13f38ce639906c9 Deleting existing unassigned node for
> 10fc985bae394d3de78d2c5595a7ee4b that is in expected state
> RS_ZK_REGION_SPLIT
> 2013-06-15 14:18:48,225 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:60000-0x13f38ce639906c9 Successfully deleted unassigned node for
> region 10fc985bae394d3de78d2c5595a7ee4b in expected state
> RS_ZK_REGION_SPLIT
> 2013-06-15 14:18:48,226 INFO
> org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handled SPLIT
> event;
>
> parent=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371276011390.10fc985bae394d3de78d2c5595a7ee4b.
> daughter
>
> a=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371277127973.060e12898838607dd55d6bb0de074af6.daughter
>
> b=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5g>\xEC,1371277127973.0584b8152c974c7b22a618a9561f303e.
> 2013-06-15 14:19:32,338 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Creating scanner over .META. starting at key ''
> 2013-06-15 14:19:32,338 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Advancing internal scanner to startKey at ''
> 2013-06-15 14:19:32,345 INFO org.apache.hadoop.hbase.master.LoadBalancer:
> Skipping load balancing because balanced cluster; servers=6 regions=890
> average=148.33333 mostloaded=168 leastloaded=135
> 2013-06-15 14:19:32,401 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Finished with scanning at {NAME => '.META.,,1', STARTKEY => '', ENDKEY =>
> '', ENCODED => 1028785192,}
> 2013-06-15 14:19:32,735 DEBUG
> org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 983 catalog row(s)
> and gc'd 0 unreferenced parent region(s)
> 2013-06-15 14:21:20,204 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:21:20,211 INFO org.apache.zookeeper.ZooKeeper: Initiating
> client connection, connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> sessionTimeout=180000
>
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:21:20,213 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server dsjhd002/132.228.248.12:2181. Will not attempt
> to authenticate using SASL (unknown error)
> 2013-06-15 14:21:20,213 INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> this process is 72221@dsjhd001
> 2013-06-15 14:21:20,213 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> 2013-06-15 14:21:20,214 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to dsjhd002/132.228.248.12:2181, initiating session
> 2013-06-15 14:21:20,219 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server dsjhd002/132.228.248.12:2181, sessionid =
> 0x23f38ce677905da, negotiated timeout = 40000
> 2013-06-15 14:21:20,225 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row= for max=2147483647 rows using
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:21:20,281 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Creating scanner over .META. starting at key 'cdr,,'
> 2013-06-15 14:21:20,281 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Advancing internal scanner to startKey at 'cdr,,'
> 2013-06-15 14:21:20,288 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Creating scanner over .META. starting at key 'cdr,,'
> 2013-06-15 14:21:20,288 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Advancing internal scanner to startKey at 'cdr,,'
> 2013-06-15 14:21:20,384 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ZooKeeper: Session:
> 0x23f38ce677905da closed
> 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ClientCnxn: EventThread
> shut down
> 2013-06-15 14:21:20,388 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row=cdr,,00000000000000 for max=2147483647 rows
> using
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:21:20,393 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,,1371219562338.f80cffbfd14f542ebf62b699ff16cd1d.
> 2013-06-15 14:21:20,394 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,,1371220461424.9a60de5a8b93cbb090f2ff638700c17f.
> 2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,,1371220844707.671b124985ca6c8ec3ec2460cdeb3a6e.
> 2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF+\x15\x8F,1371220844707.b8e125158cbe648f65975b7a55a64928.
> 2013-06-15 14:21:20,397 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFFn\xE9U,1371220461424.c2829cb530417f67beabb0e0452e72aa.
> 2013-06-15 14:21:20,399 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xA0\x87\x07,1371219562338.400457e7a191aa448aebb416306d99a8.
> 2013-06-15 14:21:20,401 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371219208991.c77fc0904715b9aa6924856bf9b822b1.
> 2013-06-15 14:21:20,402 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371221923804.d5f97df0277012c82585365c653752e1.
> 2013-06-15 14:21:20,404 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371266084527.e63be214991f76f051ea86ea9c1e3c92.
> 2013-06-15 14:21:20,406 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371268929218.fcf39c8f746f710b2412370635e931b3.
> 2013-06-15 14:21:20,407 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371269885965.25544f24893033cc60a1f4dd498fb4ec.
> 2013-06-15 14:21:20,409 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371270770875.242bf8d20a3718b3398297c38396d2c6.
> 2013-06-15 14:21:20,411 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371271313678.1b796f461937f79c2491742c3672f487.
> 2013-06-15 14:21:20,412 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371272501819.9c2db71ef235a396af00a40c2c60a30c.
> 2013-06-15 14:21:20,414 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274172249.b92c0cde8329b6d0ae9515661b1ffdd7.
> 2013-06-15 14:21:20,416 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274465375.88a56b9df947ca156d2f12e680fbf8fc.
> 2013-06-15 14:21:20,418 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
>
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371276503503.2931a51dba76afdd4d1b1b69470658f9.
>
>
> On Sat, Jun 15, 2013 at 11:05 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Have you checked master / region server logs w.r.t. such region ?
> >
> > Cheers
> >
> > On Sat, Jun 15, 2013 at 8:00 AM, hua beatls <be...@gmail.com> wrote:
> >
> > > HI,
> > > we use CDH 4.2.0, find that some of the regions stop spliting and
> > > increase contiusuly under heavy load. some regions become much bigger
> > than
> > > the setting value(4G),reaches 46g.
> > > any reason or bugs?
> > >
> > > Thansk!
> > >
> > > beatls
> > >
> >
>
Re: WHY REGIONS STOP SPLITTING
Posted by hua beatls <be...@gmail.com>.
HI,
below is part of hmaster'log:
ERROR: Region { meta =>
receipt,0000000000000000000021140012701002,1371294540657.e394dc54904e49eae921157ae0f28d0b.,
hdfs => hdfs://mycluster/hbase/receipt/e394dc54904e49eae921157ae0f28d0b,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF4\xEE\xE4\x7F\xFF\xFF\xFF\xFF\xFDS\x95,1371290397114.e5e75e98b93f3f07341414b703e84927.,
hdfs => hdfs://mycluster/hbase/cdr/e5e75e98b93f3f07341414b703e84927,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371266084527.e63be214991f76f051ea86ea9c1e3c92.,
hdfs => hdfs://mycluster/hbase/cdr/e63be214991f76f051ea86ea9c1e3c92,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF7\x91\xFE\x18,1371273491978.e96f477884e86010324008fecb92f6cb.,
hdfs => hdfs://mycluster/hbase/cdr/e96f477884e86010324008fecb92f6cb,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF5f\xE0\x7F\xFF\xFF\xFF\xF9jN\x1F,1371285891303.eb1b683bdb0bcbb9016b7ac1c6018444.,
hdfs => hdfs://mycluster/hbase/cdr/eb1b683bdb0bcbb9016b7ac1c6018444,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF5\xF7\xBE\x7F\xFF\xFF\xFF\xFF\xFE\xDB\xC9,1371275634073.f0982631a934fc01f17b24d1ebfd44f1.,
hdfs => hdfs://mycluster/hbase/cdr/f0982631a934fc01f17b24d1ebfd44f1,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF5\xF7\xBE\x7F\xFF\xFF\xFF\xFF\xFE\xDB\xC9,1371278499584.f1c3417777ca4def6e828432b51ef001.,
hdfs => hdfs://mycluster/hbase/cdr/f1c3417777ca4def6e828432b51ef001,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF4\xEE\xE4\x7F\xFF\xFF\xFF\xF5\x9BU\xCD,1371270239580.f2193b6d72f1456c1f5e94f7e05ee506.,
hdfs => hdfs://mycluster/hbase/cdr/f2193b6d72f1456c1f5e94f7e05ee506,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371281568562.f3a98cb51d0263690fed14f127a3ca2d.,
hdfs => hdfs://mycluster/hbase/cdr/f3a98cb51d0263690fed14f127a3ca2d,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF5\xF7\xBE\x7F\xFF\xFF\xFF\xF6V*\xE1,1371297543196.f764a8b78c5fd0ccccc6ca85248f7909.,
hdfs => hdfs://mycluster/hbase/cdr/f764a8b78c5fd0ccccc6ca85248f7909,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371268929218.fcf39c8f746f710b2412370635e931b3.,
hdfs => hdfs://mycluster/hbase/cdr/fcf39c8f746f710b2412370635e931b3,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
ERROR: Region { meta =>
receipt,0000000000000000000011002090791002,1371292962113.fffa2c8fc406cc88fe2acd204b1797d2.,
hdfs => hdfs://mycluster/hbase/receipt/fffa2c8fc406cc88fe2acd204b1797d2,
deployed => } is a split parent in META, in HDFS, and not deployed on any
region server. This could be transient.
13/06/16 00:51:58 DEBUG util.HBaseFsck: There are 1381 region info entries
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/dfb68f612e2bf2a59a70727652341e31/f/6475f9d34e3d405f8a317b7fc92c642e.286d1a13b57560b55b9cab25ea1b96f7'
to region=286d1a13b57560b55b9cab25ea1b96f7
hfile=6475f9d34e3d405f8a317b7fc92c642e
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/T21_0510_201303_bigtable/9cfc5d33b25bb93b38c1d2daae9663df/cf/4e884863e357438e8f4ad2bf003c8cf5.915bb1b602bf69149323e8c77db913de'
to region=915bb1b602bf69149323e8c77db913de
hfile=4e884863e357438e8f4ad2bf003c8cf5
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/de25bdb1c55db862c708027b83192633/f/3b28fe3e607442258f2800e9bb6aba07.e5e75e98b93f3f07341414b703e84927'
to region=e5e75e98b93f3f07341414b703e84927
hfile=3b28fe3e607442258f2800e9bb6aba07
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/136797e4d22668568e39e8eb41c2f769/cf/c328db0a24164520a19bcd42e85bf416.55641e5c6045f63df83eb4131876337c'
to region=55641e5c6045f63df83eb4131876337c
hfile=c328db0a24164520a19bcd42e85bf416
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/84ac03743da616eef687bf30972a1d04/f/8f9949e063a0449eb51f0d39ea5c960f.48e6c4274767b76a73d294ec8a0a7b1d'
to region=48e6c4274767b76a73d294ec8a0a7b1d
hfile=8f9949e063a0449eb51f0d39ea5c960f
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/4149f107fb6f6c4c10a7bb5dff086b42/f/e9d5c3d39da74d97abed268ea001946a.1c87d162168823a20767f0693857130a'
to region=1c87d162168823a20767f0693857130a
hfile=e9d5c3d39da74d97abed268ea001946a
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/a904f46efbdef6c7ef77f874c76661f8/f/69d2ec08ed4f4c58b5f30272fc646806.5429a1c3961b0938082b242e3cf7bfc8'
to region=5429a1c3961b0938082b242e3cf7bfc8
hfile=69d2ec08ed4f4c58b5f30272fc646806
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/a5d6d6caf37736f08750910de53f39f8/cf/2738c39aa963417f9bf07afe880104a1.190f43d3cfc7469004ce284d90325fb5'
to region=190f43d3cfc7469004ce284d90325fb5
hfile=2738c39aa963417f9bf07afe880104a1
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/c8bc8b174e7c880087fb349e5de9a730/cf/610be13da3a54fcea25c002eb7d1f088.e394dc54904e49eae921157ae0f28d0b'
to region=e394dc54904e49eae921157ae0f28d0b
hfile=610be13da3a54fcea25c002eb7d1f088
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/39fe8acdc7f6dc98ac4c8dd182ee4f77/cf/07d5a305929944b09574b4a4af5366b7.d0292859c3c717f4bfd15305250277c1'
to region=d0292859c3c717f4bfd15305250277c1
hfile=07d5a305929944b09574b4a4af5366b7
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/b2c1ccb57905e93259bab31e2b60715b/f/168e7f18eaeb44f9898424919b5faaa1.d0cfbf9789a8ab5010a450629104b9be'
to region=d0cfbf9789a8ab5010a450629104b9be
hfile=168e7f18eaeb44f9898424919b5faaa1
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/65971620ae24d343c42588d2b74f5923/f/f089d73b83be4760953dc1ec4cb14bfb.0045af9c05947dd4b946e9700b46de7f'
to region=0045af9c05947dd4b946e9700b46de7f
hfile=f089d73b83be4760953dc1ec4cb14bfb
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/e82223081c249154eafb9f38345e93f8/f/d14920964de54672b10742e5e8fb5567.8b23436f44ef6bb4eb1036b296015e8d'
to region=8b23436f44ef6bb4eb1036b296015e8d
hfile=d14920964de54672b10742e5e8fb5567
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/cb29f35c16beae1a61084b563d8f43fa/f/959d7306ab5f4005b3298f2c69c590b0.75a29c39290c38b349338a9b76f2682d'
to region=75a29c39290c38b349338a9b76f2682d
hfile=959d7306ab5f4005b3298f2c69c590b0
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/66bc4c56f93da9ebba06861411f626b0/f/35be0a0835554b5992c3ae22b4cad819.d508ac045fb11810ee73794c808df935'
to region=d508ac045fb11810ee73794c808df935
hfile=35be0a0835554b5992c3ae22b4cad819
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/324153c1e713e8108fec661616ef08c6/cf/bc05d9c0c64a404794140c10689b2508.cb19cceeb23565f6d40b53606b32952b'
to region=cb19cceeb23565f6d40b53606b32952b
hfile=bc05d9c0c64a404794140c10689b2508
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/c5339a0b6f914b3ea20981f536a69de4.031ef47e5e5792c2e3e6ec3d54df4474'
to region=031ef47e5e5792c2e3e6ec3d54df4474
hfile=c5339a0b6f914b3ea20981f536a69de4
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/d96080ea1768c4aef8046141d7c3cdcd/f/88f1cd1ee7aa454ca96966167d56b76c.21986d0838d30af14859fab9843cdc9e'
to region=21986d0838d30af14859fab9843cdc9e
hfile=88f1cd1ee7aa454ca96966167d56b76c
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/97b1e1e5f620abf15f9d82cdb096c799/f/db0e1dd561e3462d97427f1d108737bb.ade716ab1b479c045c938054d221fec8'
to region=ade716ab1b479c045c938054d221fec8
hfile=db0e1dd561e3462d97427f1d108737bb
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/b72a5117197dc83b8ee092ab754db80a/f/52e30b58fb5c4ba49c5d9037bb2daf4a.e63be214991f76f051ea86ea9c1e3c92'
to region=e63be214991f76f051ea86ea9c1e3c92
hfile=52e30b58fb5c4ba49c5d9037bb2daf4a
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/a5d6d6caf37736f08750910de53f39f8/cf/66ca237459ea4832a4b933a4204e92b1.190f43d3cfc7469004ce284d90325fb5'
to region=190f43d3cfc7469004ce284d90325fb5
hfile=66ca237459ea4832a4b933a4204e92b1
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/cee9cd4a8b93ec5e4c738358ff08251a/f/fee99309f9d84e82858f00deb6f3cbf7.a0cff34902905692f0d735bc46ea1a34'
to region=a0cff34902905692f0d735bc46ea1a34
hfile=fee99309f9d84e82858f00deb6f3cbf7
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/54c2de7d9e7be0562b7555c91c8d3f85/f/b4e9b592837d4d0d8adf9c2b7e6e501d.4afffdccd2bcefb768725c38cc937404'
to region=4afffdccd2bcefb768725c38cc937404
hfile=b4e9b592837d4d0d8adf9c2b7e6e501d
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/47a5ee20d4d44449107534c71fd280e2/f/fc50cfd6e9624b5aab9e733a43609efb.472c1fcae812432b04fc74bbcb89e10d'
to region=472c1fcae812432b04fc74bbcb89e10d
hfile=fc50cfd6e9624b5aab9e733a43609efb
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/c4f7f9ebf9595b59c2707da1867ab0b7/f/d64049011ddc4cd2a76e45d084dff005.7471f78a253b1e1161126d3d146cd625'
to region=7471f78a253b1e1161126d3d146cd625
hfile=d64049011ddc4cd2a76e45d084dff005
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/5a819fee50a06d0c2132e8b9c83a2265/f/8972edd53388443098f8b5b84cdd3e6b.8305dcba22221ef65be4c035f8185c23'
to region=8305dcba22221ef65be4c035f8185c23
hfile=8972edd53388443098f8b5b84cdd3e6b
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/f9a5734da8aa61ee88b4e9126a399178/f/0372dec58e7d425c801f8c27f7afc628.c31e0836bddb87b12a1e36f8f05cee52'
to region=c31e0836bddb87b12a1e36f8f05cee52
hfile=0372dec58e7d425c801f8c27f7afc628
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/5fa2721bd98816daef131b6d56aa126c/f/1722928e563b46f7a03465166488fe19.b53f2e460f85e28f363af26269e148b6'
to region=b53f2e460f85e28f363af26269e148b6
hfile=1722928e563b46f7a03465166488fe19
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/68dd8f4cfb183127ba2172ed0a286dc5/f/c1e63b4b07184b17ae43f8fe9e270eec.2389b49be7bc9e7ae6062a2b6dfe4498'
to region=2389b49be7bc9e7ae6062a2b6dfe4498
hfile=c1e63b4b07184b17ae43f8fe9e270eec
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/3b04ec5bd794cac6e94b5a34f9b15140/f/98018fdf7d8642c893140d136145ab71.f764a8b78c5fd0ccccc6ca85248f7909'
to region=f764a8b78c5fd0ccccc6ca85248f7909
hfile=98018fdf7d8642c893140d136145ab71
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/39fe8acdc7f6dc98ac4c8dd182ee4f77/cf/ab489c6dfea1411e9cf5ebc6b2fb79e7.d0292859c3c717f4bfd15305250277c1'
to region=d0292859c3c717f4bfd15305250277c1
hfile=ab489c6dfea1411e9cf5ebc6b2fb79e7
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/cb29f35c16beae1a61084b563d8f43fa/f/1a32ac7144474cfeb7a50bc10b9cfef5.75a29c39290c38b349338a9b76f2682d'
to region=75a29c39290c38b349338a9b76f2682d
hfile=1a32ac7144474cfeb7a50bc10b9cfef5
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/53035d321765a3d10823d4a2d5346baf/cf/82b116f6d8ac4ccdacadcefab9518852.2e39e1e9557d08a677669fe60a9f6644'
to region=2e39e1e9557d08a677669fe60a9f6644
hfile=82b116f6d8ac4ccdacadcefab9518852
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/65971620ae24d343c42588d2b74f5923/f/33340064e723472283b1acc6489a34dd.0045af9c05947dd4b946e9700b46de7f'
to region=0045af9c05947dd4b946e9700b46de7f
hfile=33340064e723472283b1acc6489a34dd
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/65971620ae24d343c42588d2b74f5923/f/59f9f7b5b11747628b0e22daade0fd30.0045af9c05947dd4b946e9700b46de7f'
to region=0045af9c05947dd4b946e9700b46de7f
hfile=59f9f7b5b11747628b0e22daade0fd30
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/7c2b369455ecd7b1d8f1e714bd3b7537/cf/c881410a1ac24c758348e686b15ef905.702b6304d12b146a1dce0b423943277e'
to region=702b6304d12b146a1dce0b423943277e
hfile=c881410a1ac24c758348e686b15ef905
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/324153c1e713e8108fec661616ef08c6/cf/de00810145e44a1cbbf78f410ff93ef5.cb19cceeb23565f6d40b53606b32952b'
to region=cb19cceeb23565f6d40b53606b32952b
hfile=de00810145e44a1cbbf78f410ff93ef5
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/b72a5117197dc83b8ee092ab754db80a/f/da39c9a99eb24686822743030eae19e3.e63be214991f76f051ea86ea9c1e3c92'
to region=e63be214991f76f051ea86ea9c1e3c92
hfile=da39c9a99eb24686822743030eae19e3
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/fa4e0232ea5310a73a4708232e7f20e0/f/8e58d89bcc56490ea904d9704c385bf4.65e995455ece73df5535c1961ecff5a4'
to region=65e995455ece73df5535c1961ecff5a4
hfile=8e58d89bcc56490ea904d9704c385bf4
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/6a25e347f1ad2d515dda5db735b5707c/f/1067a2c9ce074cb99f01f60ef73601d2.2f45a07b5043c7d097ccaecd8f0d9c45'
to region=2f45a07b5043c7d097ccaecd8f0d9c45
hfile=1067a2c9ce074cb99f01f60ef73601d2
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/f9a5734da8aa61ee88b4e9126a399178/f/cfa7c47c856244e1a494504dd65df567.c31e0836bddb87b12a1e36f8f05cee52'
to region=c31e0836bddb87b12a1e36f8f05cee52
hfile=cfa7c47c856244e1a494504dd65df567
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/e486cdfa162b807d17da83b4c8632bb5/cf/f403d05294f34100acafd41a6d0721e8.25b30cc1a09f060843e454f780e0a7dc'
to region=25b30cc1a09f060843e454f780e0a7dc
hfile=f403d05294f34100acafd41a6d0721e8
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/5244e91050452f3d99d1756fc6c2d6c1/cf/d42a4802c604477cbd41c56c95e7ba42.02fc6e651a1235f12a02cddea5cf2775'
to region=02fc6e651a1235f12a02cddea5cf2775
hfile=d42a4802c604477cbd41c56c95e7ba42
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/cb29f35c16beae1a61084b563d8f43fa/f/0fc7992a1cb34d7c941fb707c40d0a7d.75a29c39290c38b349338a9b76f2682d'
to region=75a29c39290c38b349338a9b76f2682d
hfile=0fc7992a1cb34d7c941fb707c40d0a7d
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/a8244ccc92c93d7e9bc4a154bfc4fa23/f/d9060faca7314c4686588cf72f50e8c8.d5f97df0277012c82585365c653752e1'
to region=d5f97df0277012c82585365c653752e1
hfile=d9060faca7314c4686588cf72f50e8c8
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/e2f210a5b97562967f5a141b2e152a6d/f/0e44d832dff74e188d2fda9ad5735605.0e601f2cbd2b20011d87892c2f69ec1a'
to region=0e601f2cbd2b20011d87892c2f69ec1a
hfile=0e44d832dff74e188d2fda9ad5735605
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/66094e8e17157d252446fafe76cf96e6/cf/5a2bf7249b1842538050c510d6762072.d28ce7fd27f0b8fe1e14de40ad6e623f'
to region=d28ce7fd27f0b8fe1e14de40ad6e623f
hfile=5a2bf7249b1842538050c510d6762072
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/receipt/fe86f065bab792a2daa55cfdc7988bca/cf/eff16004268c4d8e8b05eabb5699f822.6805f9fffb6384fc6befe8f4ef2c8478'
to region=6805f9fffb6384fc6befe8f4ef2c8478
hfile=eff16004268c4d8e8b05eabb5699f822
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/f02be681716d915add8563cff452ecc9/f/c44befbab290439cb1c5d0f90e75ac5f.9107a1e1f849627f2e64ac98610ec294'
to region=9107a1e1f849627f2e64ac98610ec294
hfile=c44befbab290439cb1c5d0f90e75ac5f
13/06/16 00:52:07 DEBUG regionserver.StoreFile: reference
'hdfs://mycluster/hbase/cdr/5a819fee50a06d0c2132e8b9c83a2265/f/fd3928a909004ff2a1582a45c1d5394f.8305dcba22221ef65be4c035f8185c23'
to region=8305dcba22221ef65be4c035f8185c23
hfile=fd3928a909004ff2a1582a45c1d5394f
Summary:
-ROOT- is okay.
Number of regions: 1
Deployed on: dsjhd005,60020,1371197423717
.META. is okay.
Number of regions: 1
Deployed on: dsjhd003,60020,1371197352320
A_QUERY_TYPE_CONFIG1 is okay.
Number of regions: 1
Deployed on: dsjhd008,60020,1371197352209
BUSI_TYPE is okay.
Number of regions: 1
Deployed on: dsjhd007,60020,1371197379841
CRM_ACTION_LOG is okay.
Number of regions: 326
Deployed on: dsjhd003,60020,1371197352320 dsjhd004,60020,1371197351960
dsjhd005,60020,1371197423717 dsjhd006,60020,1371197351220
dsjhd007,60020,1371197379841 dsjhd008,60020,1371197352209
CRM_ALERT_LOG is okay.
Number of regions: 1
Deployed on: dsjhd006,60020,1371197351220
CRM_AUDIT_LOG is okay.
Number of regions: 11
Deployed on: dsjhd003,60020,1371197352320 dsjhd004,60020,1371197351960
dsjhd005,60020,1371197423717 dsjhd008,60020,1371197352209
CdrBatch is okay.
Number of regions: 1
Deployed on: dsjhd003,60020,1371197352320
Hbase_Web_Log is okay.
Number of regions: 1
Deployed on: dsjhd004,60020,1371197351960
MSG_ACCT_ITEM_TYPE is okay.
Number of regions: 1
Deployed on: dsjhd008,60020,1371197352209
T1_0510_201303_bigtable is okay.
Number of regions: 6
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
T1_0511_201303_bigtable is okay.
Number of regions: 6
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
T1_0512_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd007,60020,1371197379841
T1_0513_201303_bigtable is okay.
Number of regions: 6
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
T1_0514_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717
T1_0515_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd008,60020,1371197352209
T1_0516_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd007,60020,1371197379841
T1_0523_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717
T21_0510_201303_bigtable is okay.
Number of regions: 13
Deployed on: dsjhd003,60020,1371197352320 dsjhd005,60020,1371197423717
dsjhd008,60020,1371197352209
T21_0511_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd006,60020,1371197351220
T21_0512_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd005,60020,1371197423717
T21_0513_201303_bigtable is okay.
Number of regions: 10
Deployed on: dsjhd007,60020,1371197379841 dsjhd008,60020,1371197352209
T21_0514_201303_bigtable is okay.
Number of regions: 10
Deployed on: dsjhd005,60020,1371197423717 dsjhd006,60020,1371197351220
T21_0515_201303_bigtable is okay.
Number of regions: 14
Deployed on: dsjhd003,60020,1371197352320 dsjhd005,60020,1371197423717
dsjhd006,60020,1371197351220 dsjhd008,60020,1371197352209
T21_0516_201303_bigtable is okay.
Number of regions: 12
Deployed on: dsjhd004,60020,1371197351960 dsjhd006,60020,1371197351220
dsjhd007,60020,1371197379841
T21_0523_201303_bigtable is okay.
Number of regions: 10
Deployed on: dsjhd003,60020,1371197352320 dsjhd008,60020,1371197352209
T22_0510_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd003,60020,1371197352320
T22_0511_201303_bigtable is okay.
Number of regions: 10
Deployed on: dsjhd003,60020,1371197352320 dsjhd004,60020,1371197351960
dsjhd005,60020,1371197423717 dsjhd006,60020,1371197351220
T22_0512_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd003,60020,1371197352320 dsjhd004,60020,1371197351960
dsjhd005,60020,1371197423717
T22_0513_201303_bigtable is okay.
Number of regions: 20
Deployed on: dsjhd003,60020,1371197352320 dsjhd004,60020,1371197351960
dsjhd005,60020,1371197423717 dsjhd006,60020,1371197351220
dsjhd008,60020,1371197352209
T22_0514_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd005,60020,1371197423717
T22_0515_201303_bigtable is okay.
Number of regions: 14
Deployed on: dsjhd003,60020,1371197352320 dsjhd005,60020,1371197423717
dsjhd006,60020,1371197351220
T22_0516_201303_bigtable is okay.
Number of regions: 11
Deployed on: dsjhd003,60020,1371197352320 dsjhd006,60020,1371197351220
T22_0523_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd005,60020,1371197423717
T23_0510_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd007,60020,1371197379841
T23_0511_201303_bigtable is okay.
Number of regions: 2
Deployed on: dsjhd007,60020,1371197379841
T23_0512_201303_bigtable is okay.
Number of regions: 2
Deployed on: dsjhd008,60020,1371197352209
T23_0513_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd008,60020,1371197352209
T23_0514_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd008,60020,1371197352209
T23_0515_201303_bigtable is okay.
Number of regions: 2
Deployed on: dsjhd005,60020,1371197423717
T23_0516_201303_bigtable is okay.
Number of regions: 2
Deployed on: dsjhd003,60020,1371197352320
T23_0523_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd008,60020,1371197352209
T24_0510_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd007,60020,1371197379841
T24_0511_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd005,60020,1371197423717 dsjhd006,60020,1371197351220
T24_0512_201303_bigtable is okay.
Number of regions: 10
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
dsjhd007,60020,1371197379841
T24_0513_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
T24_0514_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd008,60020,1371197352209
T24_0515_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd003,60020,1371197352320 dsjhd004,60020,1371197351960
T24_0516_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717
T24_0523_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd003,60020,1371197352320
T2_0510_201303_bigtable is okay.
Number of regions: 7
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
T2_0511_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd006,60020,1371197351220
T2_0512_201303_bigtable is okay.
Number of regions: 8
Deployed on: dsjhd003,60020,1371197352320 dsjhd006,60020,1371197351220
dsjhd008,60020,1371197352209
T2_0513_201303_bigtable is okay.
Number of regions: 7
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
T2_0514_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd007,60020,1371197379841
T2_0515_201303_bigtable is okay.
Number of regions: 6
Deployed on: dsjhd003,60020,1371197352320 dsjhd006,60020,1371197351220
T2_0516_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd006,60020,1371197351220 dsjhd007,60020,1371197379841
T2_0523_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd004,60020,1371197351960
T3_0510_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd007,60020,1371197379841
T3_0511_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd008,60020,1371197352209
T3_0512_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd003,60020,1371197352320 dsjhd005,60020,1371197423717
T3_0513_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
T3_0514_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd008,60020,1371197352209
T3_0515_201303_bigtable is okay.
Number of regions: 2
Deployed on: dsjhd005,60020,1371197423717
T3_0516_201303_bigtable is okay.
Number of regions: 2
Deployed on: dsjhd003,60020,1371197352320
T3_0523_201303_bigtable is okay.
Number of regions: 4
Deployed on: dsjhd005,60020,1371197423717 dsjhd008,60020,1371197352209
TG_PUT_ALERT is okay.
Number of regions: 1
Deployed on: dsjhd008,60020,1371197352209
TL_PUT_ACTION_LOG is okay.
Number of regions: 1
Deployed on: dsjhd003,60020,1371197352320
TL_PUT_INDB_LOG is okay.
Number of regions: 1
Deployed on: dsjhd007,60020,1371197379841
TestTable is okay.
Number of regions: 32
Deployed on: dsjhd007,60020,1371197379841
cdr is okay.
Number of regions: 110
Deployed on: dsjhd004,60020,1371197351960 dsjhd005,60020,1371197423717
dsjhd006,60020,1371197351220 dsjhd007,60020,1371197379841
dsjhd008,60020,1371197352209
merge_bigtable is okay.
Number of regions: 1
Deployed on: dsjhd007,60020,1371197379841
receipt is okay.
Number of regions: 431
Deployed on: dsjhd003,60020,1371197352320 dsjhd004,60020,1371197351960
dsjhd005,60020,1371197423717 dsjhd006,60020,1371197351220
dsjhd007,60020,1371197379841 dsjhd008,60020,1371197352209
testhua111 is okay.
Number of regions: 1
Deployed on: dsjhd004,60020,1371197351960
testhua1111 is okay.
Number of regions: 1
Deployed on: dsjhd005,60020,1371197423717
109 inconsistencies detected.
Status: INCONSISTENT
[hadoop@dsjhd001 /app/hb
On Sun, Jun 16, 2013 at 12:05 AM, Ted Yu <yu...@gmail.com> wrote:
> 2013-06-15 23:18:22,106 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/
> c5339a0b6f914b3ea20981f536a69de4.031ef47e5e5792c2e3e6ec3d54df44
> 74-hdfs://mycluster/hbase/receipt/031ef47e5e5792c2e3e6ec3d54df4474/cf/
> c5339a0b6f914b3ea20981f536a69de4-bottom
> is not splittable
>
> The above log showed that the presence of
> c5339a0b6f914b3ea20981f536a69de4-bottom,
> a reference file, prevented the region from splitting.
>
> On Sat, Jun 15, 2013 at 8:22 AM, hua beatls <be...@gmail.com> wrote:
>
> > below is the region server's log:
> > too many flush and compaction in these regionservers, below is excerpt
> > from one of the regionserver's log:
> >
> > 2013-06-15 23:17:30,719 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
> > free to archive original file.
> > 2013-06-15 23:17:30,729 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374
> > 2013-06-15 23:17:30,729 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Completed compaction of 3 file(s) in cf of
> >
> >
> receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.
> > into 6389524ff5784370a7d5e3c5f4cacfec, size=541.1m; total size for store
> is
> > 2.3g
> > 2013-06-15 23:17:30,729 INFO
> > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> > completed compaction:
> >
> >
> regionName=receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.,
> > storeName=cf, fileCount=3, fileSize=541.1m, priority=8,
> > time=1726303500804089; duration=20sec
> > 2013-06-15 23:17:30,729 DEBUG
> > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> > CompactSplitThread status: compaction_queue=(39:1), split_queue=0
> > 2013-06-15 23:17:30,730 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Starting compaction on cf in region
> >
> >
> receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
> > 2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Starting compaction of 3 file(s) in cf of
> >
> >
> receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
> > into
> >
> >
> tmpdir=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp,
> > seqid=332549270, totalSize=300.9m
> > 2013-06-15 23:17:30,730 DEBUG
> > org.apache.hadoop.hbase.regionserver.Compactor: Compacting
> >
> >
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> > keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
> > 2013-06-15 23:17:30,730 DEBUG
> > org.apache.hadoop.hbase.regionserver.Compactor: Compacting
> >
> >
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> > keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
> > 2013-06-15 23:17:30,730 DEBUG
> > org.apache.hadoop.hbase.regionserver.Compactor: Compacting
> >
> >
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
> > keycount=1217, bloomtype=NONE, size=45.5m, encoding=NONE
> > 2013-06-15 23:17:30,977 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> > Creating
> >
> >
> file=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
> > with permission=rwxrwxrwx
> > 2013-06-15 23:17:30,988 DEBUG
> > org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> > CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> > [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> > [cacheEvictOnClose=false] [cacheCompressed=false]
> > 2013-06-15 23:17:30,988 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom
> filter
> > type for
> >
> >
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1:
> > CompoundBloomFilterWriter
> > 2013-06-15 23:17:40,944 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> > DeleteFamily was added to HFile
> >
> >
> (hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1)
> >
> > 2013-06-15 23:17:40,955 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Renaming compacted file at
> >
> >
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
> > to
> >
> >
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/a0739ed1108a4a0dbf5055169b9c2ce1
> > 2013-06-15 23:17:40,977 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > Removing store files after compaction...
> > 2013-06-15 23:17:40,983 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving compacted store files.
> > 2013-06-15 23:17:40,983 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Starting to archive files:[class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> > class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> > class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40]
> > 2013-06-15 23:17:40,983 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > moving files to the archive directory:
> >
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf
> > 2013-06-15 23:17:40,985 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
> > 2013-06-15 23:17:40,986 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> > free to archive original file.
> > 2013-06-15 23:17:40,994 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
> > 2013-06-15 23:17:40,994 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50
> > 2013-06-15 23:17:40,995 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> > free to archive original file.
> > 2013-06-15 23:17:41,002 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50
> > 2013-06-15 23:17:41,002 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40
> > 2013-06-15 23:17:41,003 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
> > free to archive original file.
> > 2013-06-15 23:17:41,010 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40
> > 2013-06-15 23:17:41,011 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Completed compaction of 3 file(s) in cf of
> >
> >
> receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
> > into a0739ed1108a4a0dbf5055169b9c2ce1, size=300.9m; total size for store
> is
> > 2.1g
> > 2013-06-15 23:17:41,011 INFO
> > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> > completed compaction:
> >
> >
> regionName=receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.,
> > storeName=cf, fileCount=3, fileSize=300.9m, priority=7,
> > time=1726323253116901; duration=10sec
> > 2013-06-15 23:17:41,011 DEBUG
> > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> > CompactSplitThread status: compaction_queue=(39:0), split_queue=0
> > 2013-06-15 23:17:49,526 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Flush requested on
> >
> >
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> > 2013-06-15 23:17:49,527 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Started memstore flush for
> >
> >
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.,
> > current region memstore size 128.1m
> > 2013-06-15 23:17:49,530 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting
> >
> >
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.,
> > commencing wait for mvcc, flushsize=134284248
> > 2013-06-15 23:17:49,530 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting, commencing flushing stores
> > 2013-06-15 23:17:49,536 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> > Creating
> >
> >
> file=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
> > with permission=rwxrwxrwx
> > 2013-06-15 23:17:49,544 DEBUG
> > org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> > CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> > [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> > [cacheEvictOnClose=false] [cacheCompressed=false]
> > 2013-06-15 23:17:49,544 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom
> filter
> > type for
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976:
> > CompoundBloomFilterWriter
> > 2013-06-15 23:17:53,011 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> > DeleteFamily was added to HFile
> >
> >
> (hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976)
> >
> > 2013-06-15 23:17:53,011 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Flushed , sequenceid=332587567, memsize=128.1m, into tmp file
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
> > 2013-06-15 23:17:53,021 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > Renaming flushed file at
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
> > to
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/3ebe00cb10714c3caa097954b80ad976
> > 2013-06-15 23:17:53,037 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Added
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/3ebe00cb10714c3caa097954b80ad976,
> > entries=3411, sequenceid=332587567, filesize=127.7m
> > 2013-06-15 23:17:53,038 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished memstore flush of ~128.1m/134284248, currentsize=1.7m/1810928
> for
> > region
> >
> >
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> > in 3511ms, sequenceid=332587567, compaction requested=true
> > 2013-06-15 23:17:53,039 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > 6dd8975cfa5d08bce38f4794e90e7135 - cf: Initiating minorcompaction
> > 2013-06-15 23:17:53,073 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > Skipped compaction of cf. Only 2 file(s) of size 255.3m have met
> > compaction criteria.
> > 2013-06-15 23:17:53,073 DEBUG
> > org.apache.hadoop.hbase.regionserver.CompactSplitThread: Not compacting
> >
> >
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> > because compaction request was cancelled
> > 2013-06-15 23:17:56,062 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Flush requested on
> >
> >
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
> > 2013-06-15 23:17:56,062 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Started memstore flush for
> >
> >
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
> > current region memstore size 128.1m
> > 2013-06-15 23:17:56,062 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting
> >
> >
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
> > commencing wait for mvcc, flushsize=134284248
> > 2013-06-15 23:17:56,062 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting, commencing flushing stores
> > 2013-06-15 23:17:56,066 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> > Creating
> >
> >
> file=hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
> > with permission=rwxrwxrwx
> > 2013-06-15 23:17:56,077 DEBUG
> > org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> > CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> > [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> > [cacheEvictOnClose=false] [cacheCompressed=false]
> > 2013-06-15 23:17:56,077 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom
> filter
> > type for
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645:
> > CompoundBloomFilterWriter
> > 2013-06-15 23:17:58,986 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Flush requested on
> >
> >
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.
> > 2013-06-15 23:17:59,826 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> > DeleteFamily was added to HFile
> >
> >
> (hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645)
> >
> > 2013-06-15 23:17:59,826 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Flushed , sequenceid=332597789, memsize=128.1m, into tmp file
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
> > 2013-06-15 23:17:59,836 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > Renaming flushed file at
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
> > to
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/a394dc28ac4a451693a9bf7d51e0a645
> > 2013-06-15 23:17:59,850 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Added
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/a394dc28ac4a451693a9bf7d51e0a645,
> > entries=3411, sequenceid=332597789, filesize=127.7m
> > 2013-06-15 23:17:59,852 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished memstore flush of ~128.1m/134284248, currentsize=3.2m/3346280
> for
> > region
> >
> >
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
> > in 3790ms, sequenceid=332597789, compaction requested=false
> > 2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5-hdfs://mycluster/hbase/receipt/63ee011a1e063dc6261cdeeecae0fbf5/cf/13b4fedf7b17494580796906e544f26d-bottom
> > is not splittable
> > 2013-06-15 23:17:59,852 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Started memstore flush for
> >
> >
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.,
> > current region memstore size 128.9m
> > 2013-06-15 23:17:59,852 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting
> >
> >
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.,
> > commencing wait for mvcc, flushsize=135110976
> > 2013-06-15 23:17:59,852 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting, commencing flushing stores
> > 2013-06-15 23:17:59,855 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> > Creating
> >
> >
> file=hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
> > with permission=rwxrwxrwx
> > 2013-06-15 23:17:59,864 DEBUG
> > org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> > CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> > [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> > [cacheEvictOnClose=false] [cacheCompressed=false]
> > 2013-06-15 23:17:59,864 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom
> filter
> > type for
> >
> >
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1:
> > CompoundBloomFilterWriter
> > 2013-06-15 23:18:03,367 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> > DeleteFamily was added to HFile
> >
> >
> (hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1)
> >
> > 2013-06-15 23:18:03,367 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Flushed , sequenceid=332604970, memsize=128.9m, into tmp file
> >
> >
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
> > 2013-06-15 23:18:03,378 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > Renaming flushed file at
> >
> >
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
> > to
> >
> >
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/cf/a412083a9ac242a58d43132bac7cbeb1
> > 2013-06-15 23:18:03,395 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Added
> >
> >
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/cf/a412083a9ac242a58d43132bac7cbeb1,
> > entries=3432, sequenceid=332604970, filesize=128.4m
> > 2013-06-15 23:18:03,398 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished memstore flush of ~128.9m/135110976, currentsize=1.3m/1377880
> for
> > region
> >
> >
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.
> > in 3546ms, sequenceid=332604970, compaction requested=false
> > 2013-06-15 23:18:12,373 DEBUG
> > org.apache.hadoop.hbase.regionserver.LogRoller: HLog roll requested
> > 2013-06-15 23:18:12,400 DEBUG
> > org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: using new
> > createWriter -- HADOOP-6840
> > 2013-06-15 23:18:12,400 DEBUG
> > org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter:
> >
> >
> Path=hdfs://mycluster/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309492374,
> > syncFs=true, hflush=true, compression=false
> > 2013-06-15 23:18:12,400 DEBUG
> > org.apache.hadoop.hbase.regionserver.wal.HLog: cleanupCurrentWriter
> > waiting for transactions to get synced total 18683011 synced till here
> > 18683000
> > 2013-06-15 23:18:12,422 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > Roll
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309446700,
> > entries=73574, filesize=255195585. for
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309492374
> > 2013-06-15 23:18:12,433 DEBUG
> > org.apache.hadoop.hbase.regionserver.wal.HLog: Found 6 hlogs to remove
> out
> > of total 34; oldest outstanding sequenceid is 330673801 from region
> > 1c5829d92a29beb415bcd2077ac21a60
> > 2013-06-15 23:18:12,433 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > moving old hlog file
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371307974037
> > whose highest sequenceid is 330267393 to
> > /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371307974037
> > 2013-06-15 23:18:12,449 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > moving old hlog file
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308018234
> > whose highest sequenceid is 330351557 to
> > /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308018234
> > 2013-06-15 23:18:12,458 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > moving old hlog file
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308079693
> > whose highest sequenceid is 330422025 to
> > /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308079693
> > 2013-06-15 23:18:12,474 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > moving old hlog file
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308122233
> > whose highest sequenceid is 330489689 to
> > /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308122233
> > 2013-06-15 23:18:12,482 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > moving old hlog file
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308164543
> > whose highest sequenceid is 330561043 to
> > /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308164543
> > 2013-06-15 23:18:12,491 INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > moving old hlog file
> >
> >
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308205474
> > whose highest sequenceid is 330632938 to
> > /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308205474
> > 2013-06-15 23:18:16,003 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Flush requested on
> >
> >
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.
> > 2013-06-15 23:18:16,004 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Started memstore flush for
> >
> >
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
> > current region memstore size 128.1m
> > 2013-06-15 23:18:16,004 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting
> >
> >
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
> > commencing wait for mvcc, flushsize=134284248
> > 2013-06-15 23:18:16,004 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished snapshotting, commencing flushing stores
> > 2013-06-15 23:18:16,007 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> > Creating
> >
> >
> file=hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
> > with permission=rwxrwxrwx
> > 2013-06-15 23:18:16,017 DEBUG
> > org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> > CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> > [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> > [cacheEvictOnClose=false] [cacheCompressed=false]
> > 2013-06-15 23:18:16,017 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom
> filter
> > type for
> >
> >
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd:
> > CompoundBloomFilterWriter
> > 2013-06-15 23:18:19,724 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> > DeleteFamily was added to HFile
> >
> >
> (hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/aface1c7cb724327b93d5d1d2d0a3237)
> >
> > 2013-06-15 23:18:19,739 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Renaming compacted file at
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/aface1c7cb724327b93d5d1d2d0a3237
> > to
> >
> >
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/aface1c7cb724327b93d5d1d2d0a3237
> > 2013-06-15 23:18:19,760 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > Removing store files after compaction...
> > 2013-06-15 23:18:19,762 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving compacted store files.
> > 2013-06-15 23:18:19,762 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Starting to archive files:[class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
> > class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
> > class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
> > class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
> > class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5]
> > 2013-06-15 23:18:19,762 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > moving files to the archive directory:
> >
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf
> > 2013-06-15 23:18:19,763 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,765 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
> > free to archive original file.
> > 2013-06-15 23:18:19,773 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,773 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,775 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
> > free to archive original file.
> > 2013-06-15 23:18:19,786 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,786 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,787 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
> > free to archive original file.
> > 2013-06-15 23:18:19,798 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,798 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,800 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
> > free to archive original file.
> > 2013-06-15 23:18:19,811 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,811 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Archiving:class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,812 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > No existing file in archive
> >
> >
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5,
> > free to archive original file.
> > 2013-06-15 23:18:19,823 DEBUG
> org.apache.hadoop.hbase.backup.HFileArchiver:
> > Finished archiving file from: class
> > org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> >
> >
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5,
> > to:
> >
> >
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5
> > 2013-06-15 23:18:19,824 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Completed major compaction of 5 file(s) in cf of
> >
> >
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
> > into aface1c7cb724327b93d5d1d2d0a3237, size=2.3g; total size for store is
> > 3.0g
> > 2013-06-15 23:18:19,824 INFO
> > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> > completed compaction:
> >
> >
> regionName=receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
> > storeName=cf, fileCount=5, fileSize=4.0g, priority=7,
> > time=1725499789501847; duration=1mins, 34sec
> > 2013-06-15 23:18:19,824 DEBUG
> > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> > CompactSplitThread status: compaction_queue=(39:0), split_queue=0
> > 2013-06-15 23:18:19,824 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Starting compaction on cf in region
> >
> >
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> > 2013-06-15 23:18:19,825 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Starting compaction of 3 file(s) in cf of
> >
> >
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> > into
> >
> >
> tmpdir=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp,
> > seqid=331203769, totalSize=1.0g
> > 2013-06-15 23:18:19,825 DEBUG
> > org.apache.hadoop.hbase.regionserver.Compactor: Compacting
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/dd0c60580a2f42ffb2529378dc792736,
> > keycount=13795, bloomtype=NONE, size=516.3m, encoding=NONE
> > 2013-06-15 23:18:19,825 DEBUG
> > org.apache.hadoop.hbase.regionserver.Compactor: Compacting
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/5afd93c11371463db2a01157de686030,
> > keycount=10233, bloomtype=NONE, size=383.0m, encoding=NONE
> > 2013-06-15 23:18:19,825 DEBUG
> > org.apache.hadoop.hbase.regionserver.Compactor: Compacting
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/376827ca88674cddabbb5fe5192e72b1,
> > keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
> > 2013-06-15 23:18:19,918 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> > Creating
> >
> >
> file=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/1e63ab1190ef49ad92b2316a06c19643
> > with permission=rwxrwxrwx
> > 2013-06-15 23:18:19,925 DEBUG
> > org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> > CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> > [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> > [cacheEvictOnClose=false] [cacheCompressed=false]
> > 2013-06-15 23:18:19,925 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom
> filter
> > type for
> >
> >
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/1e63ab1190ef49ad92b2316a06c19643:
> > CompoundBloomFilterWriter
> > 2013-06-15 23:18:22,023 INFO
> > org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> > DeleteFamily was added to HFile
> >
> >
> (hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd)
> >
> > 2013-06-15 23:18:22,026 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Flushed , sequenceid=332629815, memsize=128.1m, into tmp file
> >
> >
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
> > 2013-06-15 23:18:22,072 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > Renaming flushed file at
> >
> >
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
> > to
> >
> >
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/4334387118ec4cc9aac9d540819148dd
> > 2013-06-15 23:18:22,096 INFO org.apache.hadoop.hbase.regionserver.Store:
> > Added
> >
> >
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/4334387118ec4cc9aac9d540819148dd,
> > entries=3411, sequenceid=332629815, filesize=127.7m
> > 2013-06-15 23:18:22,106 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Finished memstore flush of ~128.1m/134284248, currentsize=4.7m/4881632
> for
> > region
> >
> >
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.
> > in 6102ms, sequenceid=332629815, compaction requested=true
> > 2013-06-15 23:18:22,106 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> >
> >
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/c5339a0b6f914b3ea20981f536a69de4.031ef47e5e5792c2e3e6ec3d54df4474-hdfs://mycluster/hbase/receipt/031ef47e5e5792c2e3e6ec3d54df4474/cf/c5339a0b6f914b3ea20981f536a69de4-bottom
> > is not splittable
> > 2013-06-15 23:18:22,107 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> > cfbb3b99de7188d847c4b9f436c8bdb4 - cf: Initiating minorcompaction
> > 2013-06-15 23:18:22,107 DEBUG
> > org.apache.hadoop.hbase.regionserver.CompactSplitThread: Large Compaction
> > requested:
> >
> >
> regionName=receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
> > storeName=cf, fileCount=4, fileSize=796.4m (413.4m, 127.7m, 127.7m,
> > 127.7m), priority=7, time=1726375653135879; Because:
> > regionserver60020.cacheFlusher; compaction_queue=(39:0), split_queue=0
> >
> >
> > On Sat, Jun 15, 2013 at 11:18 PM, hua beatls <be...@gmail.com> wrote:
> >
> > > hi, TED,
> > > below is part of log from master server:
> > > 2013-06-15 14:18:18,262 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> > > 2013-06-15 14:18:26,147 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:26,153 INFO org.apache.zookeeper.ZooKeeper: Initiating
> > > client connection,
> > connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> > > sessionTimeout=180000
> > >
> >
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > > 2013-06-15 14:18:26,155 INFO
> > > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier
> of
> > > this process is 72221@dsjhd001
> > > 2013-06-15 14:18:26,155 DEBUG
> > > org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog
> tracker
> > > org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> > > 2013-06-15 14:18:26,155 INFO org.apache.zookeeper.ClientCnxn: Opening
> > > socket connection to server dsjhd003/132.228.248.13:2181. Will not
> > > attempt to authenticate using SASL (unknown error)
> > > 2013-06-15 14:18:26,157 INFO org.apache.zookeeper.ClientCnxn: Socket
> > > connection established to dsjhd003/132.228.248.13:2181, initiating
> > session
> > > 2013-06-15 14:18:26,168 INFO org.apache.zookeeper.ClientCnxn: Session
> > > establishment complete on server dsjhd003/132.228.248.13:2181,
> sessionid
> > > = 0x33f38cfddfd0598, negotiated timeout = 40000
> > > 2013-06-15 14:18:26,172 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > Scanning .META. starting at row= for max=2147483647 rows using
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > > 2013-06-15 14:18:26,210 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> > .META.
> > > starting at key 'receipt,,'
> > > 2013-06-15 14:18:26,210 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal
> scanner
> > to
> > > startKey at 'receipt,,'
> > > 2013-06-15 14:18:26,213 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> > .META.
> > > starting at key 'receipt,,'
> > > 2013-06-15 14:18:26,213 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal
> scanner
> > to
> > > startKey at 'receipt,,'
> > > 2013-06-15 14:18:26,473 DEBUG
> > > org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog
> tracker
> > > org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> > > 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ZooKeeper: Session:
> > > 0x33f38cfddfd0598 closed
> > > 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ClientCnxn:
> EventThread
> > > shut down
> > > 2013-06-15 14:18:26,485 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > Scanning .META. starting at row=receipt,,00000000000000 for
> > max=2147483647
> > > rows using
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > > 2013-06-15 14:18:26,491 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001110007251002,1371260257669.f0f8b131fb77efb81f16eb01e493953f.
> > > 2013-06-15 14:18:26,492 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001110007251002,1371261674696.a9d0635ea62ea83eb847e2dc49c1c3a3.
> > > 2013-06-15 14:18:26,493 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001111085351002,1371261674696.5f600129659614930a10215e81b551d7.
> > > 2013-06-15 14:18:26,494 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001112163491002,1371260257669.c0358d1d04b4080d32199262c835a1fa.
> > > 2013-06-15 14:18:26,495 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001130002641002,1371260606344.c6eb5c6d48c5a2b2ae7612a287273108.
> > > 2013-06-15 14:18:26,496 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001131034981002,1371260606344.221332a360f91bb525a39dfdbe47a8b3.
> > > 2013-06-15 14:18:26,497 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001132067341002,1371260431691.5469992ccbb963ef7d27f53c2e05c88d.
> > > 2013-06-15 14:18:26,497 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001141017741002,1371260431691.61fada90a01436cce4342baae8b1a341.
> > > 2013-06-15 14:18:26,499 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001190035581002,1371178883452.cf8f2ad83efefa816340cd920bc3bbf8.
> > > 2013-06-15 14:18:26,499 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001210077421002,1371178883452.d8de7e1c6e7568c76a5337e3c53162fc.
> > > 2013-06-15 14:18:26,500 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001210077421002,1371211272356.2b646d6d9433dc78315756d491ac8e63.
> > > 2013-06-15 14:18:26,501 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001210077421002,1371213862480.3c695d05eaf2342e015149947dfccce2.
> > > 2013-06-15 14:18:26,502 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001210077421002,1371215646288.bc0ae5fea3f27f66e87a6e6960e6ea22.
> > > 2013-06-15 14:18:26,503 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001210854261002,1371215646288.c754faa28ce47cb12c72ecbbc1f7c85d.
> > > 2013-06-15 14:18:26,504 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001211631141002,1371213862480.9ac7561a2297b5723c9bde2ac82a275b.
> > > 2013-06-15 14:18:26,505 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001211631141002,1371215537292.c86caaeca975e84508593176d2403726.
> > > 2013-06-15 14:18:26,505 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001220360631002,1371215537292.97757f6b0418dea7294af03a3fedfac1.
> > > 2013-06-15 14:18:26,506 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001221137531002,1371211272356.79edc478ea385978cd954025dac8b9f9.
> > > 2013-06-15 14:18:26,507 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001221137531002,1371213598407.9b1bbd5cd11099c5fedfd95bab0915df.
> > > 2013-06-15 14:18:26,508 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001221137531002,1371215427123.357463550f00d07ffb25df0d84e0b837.
> > > 2013-06-15 14:18:26,509 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001221914391002,1371215427123.71d31890d25ad0b6677a705cc7d71c37.
> > > 2013-06-15 14:18:26,510 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001230648541002,1371213598407.e6bb294918d09dc08fd309807e208a2a.
> > > 2013-06-15 14:18:26,511 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001230648541002,1371215306956.0c4e3c9a1cedcd40cca686a8b70bd980.
> > > 2013-06-15 14:18:26,512 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001231425401002,1371215306956.283bada01f51766d0280219822c99a69.
> > > 2013-06-15 14:18:26,514 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001280031821002,1371178911107.c0f3a84ffb0c39ed368941f6671d0cfd.
> > > 2013-06-15 14:18:26,514 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001300080491002,1371178911107.27b811538f6ee109f267653f5f560b87.
> > > 2013-06-15 14:18:26,515 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001330007771002,1371181427502.f6b2040f8a16b60107d71e32a810a371.
> > > 2013-06-15 14:18:26,516 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001330007771002,1371207552053.add80b8254579a34fbbf070c0bb605bd.
> > > 2013-06-15 14:18:26,517 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001331064311002,1371207552053.fe632f34b69579eccbd8b4e813b9088d.
> > > 2013-06-15 14:18:26,518 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001340042061002,1371181427502.747aa41330a84f7d5fe696defc10f985.
> > > 2013-06-15 14:18:26,519 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001380041871002,1371180369431.e390609ca6beb9b7baf8f2b4796f9dda.
> > > 2013-06-15 14:18:26,520 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001390138251002,1371180369431.23d67b78df8fb952861b6d60073b643a.
> > > 2013-06-15 14:18:26,521 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001390138251002,1371210229806.6f0cd39d615be35106aeec04c526fcda.
> > > 2013-06-15 14:18:26,522 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001390138251002,1371212995795.4f9df026d5b9026f4282edcc4f54a1ef.
> > > 2013-06-15 14:18:26,523 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001391127371002,1371212995795.a62a6b8d13925f98ce7b9243b19b3d25.
> > > 2013-06-15 14:18:26,524 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001400098641002,1371210229806.55941435b45764299ec5b6563ecf5d4b.
> > > 2013-06-15 14:18:26,525 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001400098641002,1371212823051.ce6a942475c3223c26544beedd04594d.
> > > 2013-06-15 14:18:26,526 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001401087781002,1371212823051.73837e8ba4a0598fecf560cb4a14b5e4.
> > > 2013-06-15 14:18:26,527 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001410067721002,1371259508101.57037bf83841725172996ef4d9c1a7ee.
> > > 2013-06-15 14:18:26,528 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001420069141002,1371259508101.9c1caf332f6325b60a087467c2b48fa9.
> > > 2013-06-15 14:18:26,529 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001420069141002,1371261843284.937d2e422a6130b82228443ac68afd02.
> > > 2013-06-15 14:18:26,529 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001421136541002,1371261843284.961e6d73f48f108fb82bcacbeb488775.
> > > 2013-06-15 14:18:26,530 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001430035471002,1371258634059.1b8e1a437453a9e4529179205c5ba625.
> > > 2013-06-15 14:18:26,531 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001440056171002,1371258634059.4e2ec5bdd3a8fcd599b1edaa4243333f.
> > > 2013-06-15 14:18:26,532 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001440056171002,1371262374869.0acf2c312d53e933d55d36d6e6558abc.
> > > 2013-06-15 14:18:26,533 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001441098211002,1371262374869.15bc8b334ff9407679903be126d0805d.
> > > 2013-06-15 14:18:26,537 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001490005171002,1371180009315.660270fd8baafce2f0607a2996fb81e9.
> > > 2013-06-15 14:18:26,538 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001490005171002,1371207839796.804b59d6e59d8d9c2506057cafa1b50f.
> > > 2013-06-15 14:18:26,539 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001490005171002,1371209785865.96ec8a1b777cf3e93eb5f7e173455698.
> > > 2013-06-15 14:18:26,540 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001490736191002,1371209785865.caf571b498f5d747c9dc77fcd290c161.
> > > 2013-06-15 14:18:26,540 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001491467231002,1371207839796.74ca8bcf079e8281a8d5403e5ac9b1f3.
> > > 2013-06-15 14:18:26,541 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001491467231002,1371209683706.b10e30402bf0637b915947c640fd7ca4.
> > > 2013-06-15 14:18:26,542 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001500716371002,1371209683706.899d40e11dc4e5930a4ad49ad14d831b.
> > > 2013-06-15 14:18:26,543 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001510010721002,1371180009315.9fd04413854e814904405e6298ea4ba4.
> > > 2013-06-15 14:18:26,545 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001600069801002,1371181237169.f21a35fac1906ace208dda777b010514.
> > > 2013-06-15 14:18:26,546 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001600069801002,1371208329446.5e56a0c7fc4ccc83d56b09abd586794c.
> > > 2013-06-15 14:18:26,546 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001600869361002,1371208329446.47ead382d396a937c1c25a18da6d7138.
> > > 2013-06-15 14:18:26,547 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001610159121002,1371181237169.186c47cc02f5258d4f86699ece9b7823.
> > > 2013-06-15 14:18:26,549 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001870007081002,1371259179922.8d20b1a06529454ec2d7f1f4247bf22e.
> > > 2013-06-15 14:18:26,550 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001890716531002,1371259179922.4b096c2c516438baa3668ce235fc9c90.
> > > 2013-06-15 14:18:26,551 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001890716531002,1371262229170.d40579d02a66226656a1e48e0aa5fef0.
> > > 2013-06-15 14:18:26,552 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001890716531002,1371263273244.72b3118398132ecf0b630a3aeaa0c588.
> > > 2013-06-15 14:18:26,553 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001900211511002,1371263273244.62bd89081bfbc0e1bfca7db214eebc8c.
> > > 2013-06-15 14:18:26,554 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001901130951002,1371262229170.658e697026a46c7a1097213c2b7dbc59.
> > > 2013-06-15 14:18:26,555 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001901130951002,1371263138497.f9e6520a49c52462e1a888bc3e62f71a.
> > > 2013-06-15 14:18:26,556 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001910587571002,1371263138497.3077fae3eeafdb11ae663bd7c6761418.
> > > 2013-06-15 14:18:26,557 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001920003561002,1371177634606.6805f9fffb6384fc6befe8f4ef2c8478.
> > > 2013-06-15 14:18:26,558 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001960038371002,1371177634606.61662ddd932e0e483adac0c0ad072d31.
> > > 2013-06-15 14:18:26,558 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001960038371002,1371179042477.7ea6904a4aca54b84f006dcffa98353e.
> > > 2013-06-15 14:18:26,559 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001960038371002,1371200342086.1f6ce38a8765a5de4dced1f616de590b.
> > > 2013-06-15 14:18:26,560 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001960038371002,1371202134689.f6385d90b1d6c3dd0fad4a5207a632cc.
> > > 2013-06-15 14:18:26,561 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001960038371002,1371203337958.f80a9a7182b7346c16b0e50565ead084.
> > > 2013-06-15 14:18:26,562 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001960966991002,1371203337958.0969455703fbd4594fba90eca1a5b8a7.
> > > 2013-06-15 14:18:26,563 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001961895651002,1371202134689.709bdc87c3f5cb60a0379bb55d9dc3f1.
> > > 2013-06-15 14:18:26,564 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001961895651002,1371203142268.293fc7628be8c32235b68914562f0134.
> > > 2013-06-15 14:18:26,564 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001962824291002,1371203142268.e6af9210314c5a4ac5e6f5a793c1b2f3.
> > > 2013-06-15 14:18:26,565 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001970486031002,1371200342086.7ea869039a9d75c2d58a36bd1e3f43e2.
> > > 2013-06-15 14:18:26,566 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001970486031002,1371201529596.c84ee1352bd7042f01c225a07ec9fda5.
> > > 2013-06-15 14:18:26,567 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001970486031002,1371202941763.6c4f92536fb9eede8da7f9a01d431b80.
> > > 2013-06-15 14:18:26,568 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001971414651002,1371202941763.969715998f0266c37e14e3ef89d3ea92.
> > > 2013-06-15 14:18:26,569 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001972343311002,1371201529596.fcdc2b301fa6d7792dd3e5535e8cc58a.
> > > 2013-06-15 14:18:26,570 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001972343311002,1371202732228.d091d6d827acc608724b10c5f2b72010.
> > > 2013-06-15 14:18:26,570 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001973271951002,1371202732228.b7dc98ba29c04b199d9c839beadfb7e6.
> > > 2013-06-15 14:18:26,571 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001980074991002,1371179042477.e8dd68c0f884a491e63f477cf2319164.
> > > 2013-06-15 14:18:26,572 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001980074991002,1371209818804.b634bc05efa2d4b6eb98f85ad4fde93d.
> > > 2013-06-15 14:18:26,573 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001980074991002,1371212403387.705ad001a08ff629c21c8038a7e87363.
> > > 2013-06-15 14:18:26,574 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001980074991002,1371214663565.b13dd722151a2ea44a6aba5d7af1905b.
> > > 2013-06-15 14:18:26,574 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001980074991002,1371216342982.25b9c7134d2baf28f85b7e588c42fa1f.
> > > 2013-06-15 14:18:26,575 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001980690071002,1371216342982.8a3ece0302a7aad860a9ce98e418d664.
> > > 2013-06-15 14:18:26,576 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001981305191002,1371214663565.7e7375671360e812eb6e7adeff39b3a2.
> > > 2013-06-15 14:18:26,577 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001981305191002,1371216258668.69aa7ba14b3748dd78a035fe0fa73994.
> > > 2013-06-15 14:18:26,578 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001981920291002,1371216258668.a9e3bd75ed74cffefe41b2622a489cda.
> > > 2013-06-15 14:18:26,579 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001982535431002,1371212403387.3364f1a6b67a83de7f0e9b62064f6b48.
> > > 2013-06-15 14:18:26,580 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001982535431002,1371214478503.a8e263e0ba0190e55da471542efb3fc9.
> > > 2013-06-15 14:18:26,580 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001982535431002,1371216173963.a7a72feecebdd2c4d21be2808b4550a8.
> > > 2013-06-15 14:18:26,581 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001990432891002,1371216173963.bf8da05c0e99900af0fbb1fc2c9a5b5a.
> > > 2013-06-15 14:18:26,582 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001991048011002,1371214478503.ea2e8bc4b27b0a1b440586a197d67d7f.
> > > 2013-06-15 14:18:26,583 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001991048011002,1371216090115.cbe8280d8b6ebbbb5386f1c34b1ab98b.
> > > 2013-06-15 14:18:26,584 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001991663111002,1371216090115.bb548f5af5e0d9c2a34e456ff23472e1.
> > > 2013-06-15 14:18:26,585 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001992278251002,1371209818804.a0e860fe725cc86184d8525d134d4344.
> > > 2013-06-15 14:18:26,587 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001992278251002,1371211949705.6b119a297db81b2177909e51d833358e.
> > > 2013-06-15 14:18:26,588 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001992278251002,1371214293670.f091e9463441e08f39e84074ab58f3f0.
> > > 2013-06-15 14:18:26,589 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000001992278251002,1371215973835.1a1e1125671301df2b54f1a0527daa18.
> > > 2013-06-15 14:18:26,589 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000002240013061002,1371215973835.42d223d52e9ecad03dd10ebe9542551e.
> > > 2013-06-15 14:18:26,590 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000002620017131002,1371214293670.a462b626169d5286149303e76eb83f52.
> > > 2013-06-15 14:18:26,591 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000002620017131002,1371215898321.aa059217f2f2788f3a23bad75ff34e26.
> > > 2013-06-15 14:18:26,592 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000011000071171002,1371215898321.3a4c342e75751ec441a2d63c3f06a5f2.
> > > 2013-06-15 14:18:26,593 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000011000686571002,1371211949705.3668041afa8e2cce5e8abb80a8d2de54.
> > > 2013-06-15 14:18:26,594 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000011000686571002,1371214112540.9a285ce01761253f977dbed2def5b953.
> > > 2013-06-15 14:18:26,594 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000011000686571002,1371215818016.2d2a6518a7bd35f0615fc990acec5238.
> > > 2013-06-15 14:18:26,595 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000011001301951002,1371215818016.799bf01965ad79f97e128f6c80938809.
> > > 2013-06-15 14:18:26,596 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000011001917351002,1371214112540.921f4f2af20f783f938223c49065b28e.
> > > 2013-06-15 14:18:26,597 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000011001917351002,1371215735171.f8b8b79d2aa33182a60f16aafe449524.
> > > 2013-06-15 14:18:26,598 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> > > 2013-06-15 14:18:29,377 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:30,761 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:36,248 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:37,049 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:37,735 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:38,416 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:39,139 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:39,933 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:40,627 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:41,285 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:42,298 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:42,949 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:43,635 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:18:47,984 DEBUG
> > > org.apache.hadoop.hbase.master.AssignmentManager: Handling
> > > transition=RS_ZK_REGION_SPLITTING, server=dsjhd006,60020,1371197351220,
> > > region=10fc985bae394d3de78d2c5595a7ee4b
> > > 2013-06-15 14:18:48,220 DEBUG
> > > org.apache.hadoop.hbase.master.AssignmentManager: Handling
> > > transition=RS_ZK_REGION_SPLIT, server=dsjhd006,60020,1371197351220,
> > > region=10fc985bae394d3de78d2c5595a7ee4b
> > > 2013-06-15 14:18:48,221 DEBUG
> > > org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handling
> SPLIT
> > > event for 10fc985bae394d3de78d2c5595a7ee4b; deleting node
> > > 2013-06-15 14:18:48,221 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZKAssign:
> > > master:60000-0x13f38ce639906c9 Deleting existing unassigned node for
> > > 10fc985bae394d3de78d2c5595a7ee4b that is in expected state
> > > RS_ZK_REGION_SPLIT
> > > 2013-06-15 14:18:48,225 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZKAssign:
> > > master:60000-0x13f38ce639906c9 Successfully deleted unassigned node for
> > > region 10fc985bae394d3de78d2c5595a7ee4b in expected state
> > RS_ZK_REGION_SPLIT
> > > 2013-06-15 14:18:48,226 INFO
> > > org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handled
> SPLIT
> > > event;
> > >
> >
> parent=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371276011390.10fc985bae394d3de78d2c5595a7ee4b.
> > > daughter
> > >
> >
> a=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371277127973.060e12898838607dd55d6bb0de074af6.daughter
> > >
> >
> b=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5g>\xEC,1371277127973.0584b8152c974c7b22a618a9561f303e.
> > > 2013-06-15 14:19:32,338 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> > .META.
> > > starting at key ''
> > > 2013-06-15 14:19:32,338 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal
> scanner
> > to
> > > startKey at ''
> > > 2013-06-15 14:19:32,345 INFO
> org.apache.hadoop.hbase.master.LoadBalancer:
> > > Skipping load balancing because balanced cluster; servers=6 regions=890
> > > average=148.33333 mostloaded=168 leastloaded=135
> > > 2013-06-15 14:19:32,401 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Finished with scanning at
> > > {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED =>
> > 1028785192,}
> > > 2013-06-15 14:19:32,735 DEBUG
> > > org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 983 catalog
> row(s)
> > > and gc'd 0 unreferenced parent region(s)
> > > 2013-06-15 14:21:20,204 WARN org.apache.hadoop.conf.Configuration:
> > > fs.default.name is deprecated. Instead, use fs.defaultFS
> > > 2013-06-15 14:21:20,211 INFO org.apache.zookeeper.ZooKeeper: Initiating
> > > client connection,
> > connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> > > sessionTimeout=180000
> > >
> >
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > > 2013-06-15 14:21:20,213 INFO org.apache.zookeeper.ClientCnxn: Opening
> > > socket connection to server dsjhd002/132.228.248.12:2181. Will not
> > > attempt to authenticate using SASL (unknown error)
> > > 2013-06-15 14:21:20,213 INFO
> > > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier
> of
> > > this process is 72221@dsjhd001
> > > 2013-06-15 14:21:20,213 DEBUG
> > > org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog
> tracker
> > > org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> > > 2013-06-15 14:21:20,214 INFO org.apache.zookeeper.ClientCnxn: Socket
> > > connection established to dsjhd002/132.228.248.12:2181, initiating
> > session
> > > 2013-06-15 14:21:20,219 INFO org.apache.zookeeper.ClientCnxn: Session
> > > establishment complete on server dsjhd002/132.228.248.12:2181,
> sessionid
> > > = 0x23f38ce677905da, negotiated timeout = 40000
> > > 2013-06-15 14:21:20,225 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > Scanning .META. starting at row= for max=2147483647 rows using
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > > 2013-06-15 14:21:20,281 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> > .META.
> > > starting at key 'cdr,,'
> > > 2013-06-15 14:21:20,281 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal
> scanner
> > to
> > > startKey at 'cdr,,'
> > > 2013-06-15 14:21:20,288 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> > .META.
> > > starting at key 'cdr,,'
> > > 2013-06-15 14:21:20,288 DEBUG
> > > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal
> scanner
> > to
> > > startKey at 'cdr,,'
> > > 2013-06-15 14:21:20,384 DEBUG
> > > org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog
> tracker
> > > org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> > > 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ZooKeeper: Session:
> > > 0x23f38ce677905da closed
> > > 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ClientCnxn:
> EventThread
> > > shut down
> > > 2013-06-15 14:21:20,388 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > Scanning .META. starting at row=cdr,,00000000000000 for max=2147483647
> > rows
> > > using
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > > 2013-06-15 14:21:20,393 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > > cdr,,1371219562338.f80cffbfd14f542ebf62b699ff16cd1d.
> > > 2013-06-15 14:21:20,394 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > > cdr,,1371220461424.9a60de5a8b93cbb090f2ff638700c17f.
> > > 2013-06-15 14:21:20,395 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > > cdr,,1371220844707.671b124985ca6c8ec3ec2460cdeb3a6e.
> > > 2013-06-15 14:21:20,395 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF+\x15\x8F,1371220844707.b8e125158cbe648f65975b7a55a64928.
> > > 2013-06-15 14:21:20,397 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFFn\xE9U,1371220461424.c2829cb530417f67beabb0e0452e72aa.
> > > 2013-06-15 14:21:20,399 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xA0\x87\x07,1371219562338.400457e7a191aa448aebb416306d99a8.
> > > 2013-06-15 14:21:20,401 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371219208991.c77fc0904715b9aa6924856bf9b822b1.
> > > 2013-06-15 14:21:20,402 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371221923804.d5f97df0277012c82585365c653752e1.
> > > 2013-06-15 14:21:20,404 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371266084527.e63be214991f76f051ea86ea9c1e3c92.
> > > 2013-06-15 14:21:20,406 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371268929218.fcf39c8f746f710b2412370635e931b3.
> > > 2013-06-15 14:21:20,407 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371269885965.25544f24893033cc60a1f4dd498fb4ec.
> > > 2013-06-15 14:21:20,409 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371270770875.242bf8d20a3718b3398297c38396d2c6.
> > > 2013-06-15 14:21:20,411 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371271313678.1b796f461937f79c2491742c3672f487.
> > > 2013-06-15 14:21:20,412 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371272501819.9c2db71ef235a396af00a40c2c60a30c.
> > > 2013-06-15 14:21:20,414 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274172249.b92c0cde8329b6d0ae9515661b1ffdd7.
> > > 2013-06-15 14:21:20,416 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274465375.88a56b9df947ca156d2f12e680fbf8fc.
> > > 2013-06-15 14:21:20,418 DEBUG
> org.apache.hadoop.hbase.client.MetaScanner:
> > > blocking until region is in META:
> > >
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371276503503.2931a51dba76afdd4d1b1b69470658f9.
> > >
> > >
> > > On Sat, Jun 15, 2013 at 11:05 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > >> Have you checked master / region server logs w.r.t. such region ?
> > >>
> > >> Cheers
> > >>
> > >> On Sat, Jun 15, 2013 at 8:00 AM, hua beatls <be...@gmail.com> wrote:
> > >>
> > >> > HI,
> > >> > we use CDH 4.2.0, find that some of the regions stop spliting
> and
> > >> > increase contiusuly under heavy load. some regions become much
> bigger
> > >> than
> > >> > the setting value(4G),reaches 46g.
> > >> > any reason or bugs?
> > >> >
> > >> > Thansk!
> > >> >
> > >> > beatls
> > >> >
> > >>
> > >
> > >
> >
>
Re: WHY REGIONS STOP SPLITTING
Posted by Ted Yu <yu...@gmail.com>.
2013-06-15 23:18:22,106 DEBUG org.apache.hadoop.hbase.regionserver.Store:
hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/
c5339a0b6f914b3ea20981f536a69de4.031ef47e5e5792c2e3e6ec3d54df44
74-hdfs://mycluster/hbase/receipt/031ef47e5e5792c2e3e6ec3d54df4474/cf/
c5339a0b6f914b3ea20981f536a69de4-bottom
is not splittable
The above log showed that the presence of
c5339a0b6f914b3ea20981f536a69de4-bottom,
a reference file, prevented the region from splitting.
On Sat, Jun 15, 2013 at 8:22 AM, hua beatls <be...@gmail.com> wrote:
> below is the region server's log:
> too many flush and compaction in these regionservers, below is excerpt
> from one of the regionserver's log:
>
> 2013-06-15 23:17:30,719 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
> free to archive original file.
> 2013-06-15 23:17:30,729 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374
> 2013-06-15 23:17:30,729 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed compaction of 3 file(s) in cf of
>
> receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.
> into 6389524ff5784370a7d5e3c5f4cacfec, size=541.1m; total size for store is
> 2.3g
> 2013-06-15 23:17:30,729 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
>
> regionName=receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.,
> storeName=cf, fileCount=3, fileSize=541.1m, priority=8,
> time=1726303500804089; duration=20sec
> 2013-06-15 23:17:30,729 DEBUG
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> CompactSplitThread status: compaction_queue=(39:1), split_queue=0
> 2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Starting compaction on cf in region
>
> receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
> 2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.Store:
> Starting compaction of 3 file(s) in cf of
>
> receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
> into
>
> tmpdir=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp,
> seqid=332549270, totalSize=300.9m
> 2013-06-15 23:17:30,730 DEBUG
> org.apache.hadoop.hbase.regionserver.Compactor: Compacting
>
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
> 2013-06-15 23:17:30,730 DEBUG
> org.apache.hadoop.hbase.regionserver.Compactor: Compacting
>
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
> 2013-06-15 23:17:30,730 DEBUG
> org.apache.hadoop.hbase.regionserver.Compactor: Compacting
>
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
> keycount=1217, bloomtype=NONE, size=45.5m, encoding=NONE
> 2013-06-15 23:17:30,977 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
>
> file=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
> with permission=rwxrwxrwx
> 2013-06-15 23:17:30,988 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2013-06-15 23:17:30,988 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
>
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1:
> CompoundBloomFilterWriter
> 2013-06-15 23:17:40,944 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
>
> (hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1)
>
> 2013-06-15 23:17:40,955 INFO org.apache.hadoop.hbase.regionserver.Store:
> Renaming compacted file at
>
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
> to
>
> hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/a0739ed1108a4a0dbf5055169b9c2ce1
> 2013-06-15 23:17:40,977 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Removing store files after compaction...
> 2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving compacted store files.
> 2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Starting to archive files:[class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40]
> 2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> moving files to the archive directory:
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf
> 2013-06-15 23:17:40,985 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
> 2013-06-15 23:17:40,986 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> free to archive original file.
> 2013-06-15 23:17:40,994 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
> 2013-06-15 23:17:40,994 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50
> 2013-06-15 23:17:40,995 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> free to archive original file.
> 2013-06-15 23:17:41,002 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50
> 2013-06-15 23:17:41,002 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40
> 2013-06-15 23:17:41,003 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
> free to archive original file.
> 2013-06-15 23:17:41,010 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40
> 2013-06-15 23:17:41,011 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed compaction of 3 file(s) in cf of
>
> receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
> into a0739ed1108a4a0dbf5055169b9c2ce1, size=300.9m; total size for store is
> 2.1g
> 2013-06-15 23:17:41,011 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
>
> regionName=receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.,
> storeName=cf, fileCount=3, fileSize=300.9m, priority=7,
> time=1726323253116901; duration=10sec
> 2013-06-15 23:17:41,011 DEBUG
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> CompactSplitThread status: compaction_queue=(39:0), split_queue=0
> 2013-06-15 23:17:49,526 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Flush requested on
>
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> 2013-06-15 23:17:49,527 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Started memstore flush for
>
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.,
> current region memstore size 128.1m
> 2013-06-15 23:17:49,530 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting
>
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.,
> commencing wait for mvcc, flushsize=134284248
> 2013-06-15 23:17:49,530 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting, commencing flushing stores
> 2013-06-15 23:17:49,536 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
>
> file=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
> with permission=rwxrwxrwx
> 2013-06-15 23:17:49,544 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2013-06-15 23:17:49,544 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976:
> CompoundBloomFilterWriter
> 2013-06-15 23:17:53,011 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
>
> (hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976)
>
> 2013-06-15 23:17:53,011 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=332587567, memsize=128.1m, into tmp file
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
> 2013-06-15 23:17:53,021 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
> to
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/3ebe00cb10714c3caa097954b80ad976
> 2013-06-15 23:17:53,037 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/3ebe00cb10714c3caa097954b80ad976,
> entries=3411, sequenceid=332587567, filesize=127.7m
> 2013-06-15 23:17:53,038 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.1m/134284248, currentsize=1.7m/1810928 for
> region
>
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> in 3511ms, sequenceid=332587567, compaction requested=true
> 2013-06-15 23:17:53,039 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> 6dd8975cfa5d08bce38f4794e90e7135 - cf: Initiating minorcompaction
> 2013-06-15 23:17:53,073 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Skipped compaction of cf. Only 2 file(s) of size 255.3m have met
> compaction criteria.
> 2013-06-15 23:17:53,073 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Not compacting
>
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> because compaction request was cancelled
> 2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Flush requested on
>
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
> 2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Started memstore flush for
>
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
> current region memstore size 128.1m
> 2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting
>
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
> commencing wait for mvcc, flushsize=134284248
> 2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting, commencing flushing stores
> 2013-06-15 23:17:56,066 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
>
> file=hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
> with permission=rwxrwxrwx
> 2013-06-15 23:17:56,077 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2013-06-15 23:17:56,077 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645:
> CompoundBloomFilterWriter
> 2013-06-15 23:17:58,986 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Flush requested on
>
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.
> 2013-06-15 23:17:59,826 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
>
> (hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645)
>
> 2013-06-15 23:17:59,826 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=332597789, memsize=128.1m, into tmp file
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
> 2013-06-15 23:17:59,836 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
> to
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/a394dc28ac4a451693a9bf7d51e0a645
> 2013-06-15 23:17:59,850 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/a394dc28ac4a451693a9bf7d51e0a645,
> entries=3411, sequenceid=332597789, filesize=127.7m
> 2013-06-15 23:17:59,852 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.1m/134284248, currentsize=3.2m/3346280 for
> region
>
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
> in 3790ms, sequenceid=332597789, compaction requested=false
> 2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.Store:
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5-hdfs://mycluster/hbase/receipt/63ee011a1e063dc6261cdeeecae0fbf5/cf/13b4fedf7b17494580796906e544f26d-bottom
> is not splittable
> 2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Started memstore flush for
>
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.,
> current region memstore size 128.9m
> 2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting
>
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.,
> commencing wait for mvcc, flushsize=135110976
> 2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting, commencing flushing stores
> 2013-06-15 23:17:59,855 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
>
> file=hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
> with permission=rwxrwxrwx
> 2013-06-15 23:17:59,864 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2013-06-15 23:17:59,864 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
>
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1:
> CompoundBloomFilterWriter
> 2013-06-15 23:18:03,367 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
>
> (hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1)
>
> 2013-06-15 23:18:03,367 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=332604970, memsize=128.9m, into tmp file
>
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
> 2013-06-15 23:18:03,378 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
>
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
> to
>
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/cf/a412083a9ac242a58d43132bac7cbeb1
> 2013-06-15 23:18:03,395 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
>
> hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/cf/a412083a9ac242a58d43132bac7cbeb1,
> entries=3432, sequenceid=332604970, filesize=128.4m
> 2013-06-15 23:18:03,398 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.9m/135110976, currentsize=1.3m/1377880 for
> region
>
> receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.
> in 3546ms, sequenceid=332604970, compaction requested=false
> 2013-06-15 23:18:12,373 DEBUG
> org.apache.hadoop.hbase.regionserver.LogRoller: HLog roll requested
> 2013-06-15 23:18:12,400 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: using new
> createWriter -- HADOOP-6840
> 2013-06-15 23:18:12,400 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter:
>
> Path=hdfs://mycluster/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309492374,
> syncFs=true, hflush=true, compression=false
> 2013-06-15 23:18:12,400 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.HLog: cleanupCurrentWriter
> waiting for transactions to get synced total 18683011 synced till here
> 18683000
> 2013-06-15 23:18:12,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
> Roll
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309446700,
> entries=73574, filesize=255195585. for
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309492374
> 2013-06-15 23:18:12,433 DEBUG
> org.apache.hadoop.hbase.regionserver.wal.HLog: Found 6 hlogs to remove out
> of total 34; oldest outstanding sequenceid is 330673801 from region
> 1c5829d92a29beb415bcd2077ac21a60
> 2013-06-15 23:18:12,433 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
> moving old hlog file
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371307974037
> whose highest sequenceid is 330267393 to
> /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371307974037
> 2013-06-15 23:18:12,449 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
> moving old hlog file
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308018234
> whose highest sequenceid is 330351557 to
> /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308018234
> 2013-06-15 23:18:12,458 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
> moving old hlog file
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308079693
> whose highest sequenceid is 330422025 to
> /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308079693
> 2013-06-15 23:18:12,474 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
> moving old hlog file
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308122233
> whose highest sequenceid is 330489689 to
> /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308122233
> 2013-06-15 23:18:12,482 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
> moving old hlog file
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308164543
> whose highest sequenceid is 330561043 to
> /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308164543
> 2013-06-15 23:18:12,491 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
> moving old hlog file
>
> /hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308205474
> whose highest sequenceid is 330632938 to
> /hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308205474
> 2013-06-15 23:18:16,003 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Flush requested on
>
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.
> 2013-06-15 23:18:16,004 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Started memstore flush for
>
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
> current region memstore size 128.1m
> 2013-06-15 23:18:16,004 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting
>
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
> commencing wait for mvcc, flushsize=134284248
> 2013-06-15 23:18:16,004 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Finished snapshotting, commencing flushing stores
> 2013-06-15 23:18:16,007 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
>
> file=hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
> with permission=rwxrwxrwx
> 2013-06-15 23:18:16,017 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2013-06-15 23:18:16,017 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
>
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd:
> CompoundBloomFilterWriter
> 2013-06-15 23:18:19,724 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
>
> (hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/aface1c7cb724327b93d5d1d2d0a3237)
>
> 2013-06-15 23:18:19,739 INFO org.apache.hadoop.hbase.regionserver.Store:
> Renaming compacted file at
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/aface1c7cb724327b93d5d1d2d0a3237
> to
>
> hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/aface1c7cb724327b93d5d1d2d0a3237
> 2013-06-15 23:18:19,760 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Removing store files after compaction...
> 2013-06-15 23:18:19,762 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving compacted store files.
> 2013-06-15 23:18:19,762 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Starting to archive files:[class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
> class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
> class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
> class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
> class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5]
> 2013-06-15 23:18:19,762 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> moving files to the archive directory:
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf
> 2013-06-15 23:18:19,763 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,765 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
> free to archive original file.
> 2013-06-15 23:18:19,773 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,773 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,775 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
> free to archive original file.
> 2013-06-15 23:18:19,786 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,786 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,787 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
> free to archive original file.
> 2013-06-15 23:18:19,798 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,798 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,800 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
> free to archive original file.
> 2013-06-15 23:18:19,811 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,811 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Archiving:class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,812 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> No existing file in archive
>
> for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5,
> free to archive original file.
> 2013-06-15 23:18:19,823 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Finished archiving file from: class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
>
> file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5,
> to:
>
> hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5
> 2013-06-15 23:18:19,824 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 5 file(s) in cf of
>
> receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
> into aface1c7cb724327b93d5d1d2d0a3237, size=2.3g; total size for store is
> 3.0g
> 2013-06-15 23:18:19,824 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
>
> regionName=receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
> storeName=cf, fileCount=5, fileSize=4.0g, priority=7,
> time=1725499789501847; duration=1mins, 34sec
> 2013-06-15 23:18:19,824 DEBUG
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> CompactSplitThread status: compaction_queue=(39:0), split_queue=0
> 2013-06-15 23:18:19,824 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Starting compaction on cf in region
>
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> 2013-06-15 23:18:19,825 INFO org.apache.hadoop.hbase.regionserver.Store:
> Starting compaction of 3 file(s) in cf of
>
> receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
> into
>
> tmpdir=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp,
> seqid=331203769, totalSize=1.0g
> 2013-06-15 23:18:19,825 DEBUG
> org.apache.hadoop.hbase.regionserver.Compactor: Compacting
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/dd0c60580a2f42ffb2529378dc792736,
> keycount=13795, bloomtype=NONE, size=516.3m, encoding=NONE
> 2013-06-15 23:18:19,825 DEBUG
> org.apache.hadoop.hbase.regionserver.Compactor: Compacting
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/5afd93c11371463db2a01157de686030,
> keycount=10233, bloomtype=NONE, size=383.0m, encoding=NONE
> 2013-06-15 23:18:19,825 DEBUG
> org.apache.hadoop.hbase.regionserver.Compactor: Compacting
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/376827ca88674cddabbb5fe5192e72b1,
> keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
> 2013-06-15 23:18:19,918 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> Creating
>
> file=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/1e63ab1190ef49ad92b2316a06c19643
> with permission=rwxrwxrwx
> 2013-06-15 23:18:19,925 DEBUG
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
> CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
> [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
> [cacheEvictOnClose=false] [cacheCompressed=false]
> 2013-06-15 23:18:19,925 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
> type for
>
> hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/1e63ab1190ef49ad92b2316a06c19643:
> CompoundBloomFilterWriter
> 2013-06-15 23:18:22,023 INFO
> org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
> DeleteFamily was added to HFile
>
> (hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd)
>
> 2013-06-15 23:18:22,026 INFO org.apache.hadoop.hbase.regionserver.Store:
> Flushed , sequenceid=332629815, memsize=128.1m, into tmp file
>
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
> 2013-06-15 23:18:22,072 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> Renaming flushed file at
>
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
> to
>
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/4334387118ec4cc9aac9d540819148dd
> 2013-06-15 23:18:22,096 INFO org.apache.hadoop.hbase.regionserver.Store:
> Added
>
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/4334387118ec4cc9aac9d540819148dd,
> entries=3411, sequenceid=332629815, filesize=127.7m
> 2013-06-15 23:18:22,106 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.1m/134284248, currentsize=4.7m/4881632 for
> region
>
> receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.
> in 6102ms, sequenceid=332629815, compaction requested=true
> 2013-06-15 23:18:22,106 DEBUG org.apache.hadoop.hbase.regionserver.Store:
>
> hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/c5339a0b6f914b3ea20981f536a69de4.031ef47e5e5792c2e3e6ec3d54df4474-hdfs://mycluster/hbase/receipt/031ef47e5e5792c2e3e6ec3d54df4474/cf/c5339a0b6f914b3ea20981f536a69de4-bottom
> is not splittable
> 2013-06-15 23:18:22,107 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> cfbb3b99de7188d847c4b9f436c8bdb4 - cf: Initiating minorcompaction
> 2013-06-15 23:18:22,107 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Large Compaction
> requested:
>
> regionName=receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
> storeName=cf, fileCount=4, fileSize=796.4m (413.4m, 127.7m, 127.7m,
> 127.7m), priority=7, time=1726375653135879; Because:
> regionserver60020.cacheFlusher; compaction_queue=(39:0), split_queue=0
>
>
> On Sat, Jun 15, 2013 at 11:18 PM, hua beatls <be...@gmail.com> wrote:
>
> > hi, TED,
> > below is part of log from master server:
> > 2013-06-15 14:18:18,262 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> > 2013-06-15 14:18:26,147 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:26,153 INFO org.apache.zookeeper.ZooKeeper: Initiating
> > client connection,
> connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> > sessionTimeout=180000
> >
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > 2013-06-15 14:18:26,155 INFO
> > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> > this process is 72221@dsjhd001
> > 2013-06-15 14:18:26,155 DEBUG
> > org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
> > org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> > 2013-06-15 14:18:26,155 INFO org.apache.zookeeper.ClientCnxn: Opening
> > socket connection to server dsjhd003/132.228.248.13:2181. Will not
> > attempt to authenticate using SASL (unknown error)
> > 2013-06-15 14:18:26,157 INFO org.apache.zookeeper.ClientCnxn: Socket
> > connection established to dsjhd003/132.228.248.13:2181, initiating
> session
> > 2013-06-15 14:18:26,168 INFO org.apache.zookeeper.ClientCnxn: Session
> > establishment complete on server dsjhd003/132.228.248.13:2181, sessionid
> > = 0x33f38cfddfd0598, negotiated timeout = 40000
> > 2013-06-15 14:18:26,172 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > Scanning .META. starting at row= for max=2147483647 rows using
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > 2013-06-15 14:18:26,210 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> .META.
> > starting at key 'receipt,,'
> > 2013-06-15 14:18:26,210 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner
> to
> > startKey at 'receipt,,'
> > 2013-06-15 14:18:26,213 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> .META.
> > starting at key 'receipt,,'
> > 2013-06-15 14:18:26,213 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner
> to
> > startKey at 'receipt,,'
> > 2013-06-15 14:18:26,473 DEBUG
> > org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
> > org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> > 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ZooKeeper: Session:
> > 0x33f38cfddfd0598 closed
> > 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ClientCnxn: EventThread
> > shut down
> > 2013-06-15 14:18:26,485 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > Scanning .META. starting at row=receipt,,00000000000000 for
> max=2147483647
> > rows using
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > 2013-06-15 14:18:26,491 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001110007251002,1371260257669.f0f8b131fb77efb81f16eb01e493953f.
> > 2013-06-15 14:18:26,492 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001110007251002,1371261674696.a9d0635ea62ea83eb847e2dc49c1c3a3.
> > 2013-06-15 14:18:26,493 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001111085351002,1371261674696.5f600129659614930a10215e81b551d7.
> > 2013-06-15 14:18:26,494 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001112163491002,1371260257669.c0358d1d04b4080d32199262c835a1fa.
> > 2013-06-15 14:18:26,495 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001130002641002,1371260606344.c6eb5c6d48c5a2b2ae7612a287273108.
> > 2013-06-15 14:18:26,496 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001131034981002,1371260606344.221332a360f91bb525a39dfdbe47a8b3.
> > 2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001132067341002,1371260431691.5469992ccbb963ef7d27f53c2e05c88d.
> > 2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001141017741002,1371260431691.61fada90a01436cce4342baae8b1a341.
> > 2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001190035581002,1371178883452.cf8f2ad83efefa816340cd920bc3bbf8.
> > 2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001210077421002,1371178883452.d8de7e1c6e7568c76a5337e3c53162fc.
> > 2013-06-15 14:18:26,500 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001210077421002,1371211272356.2b646d6d9433dc78315756d491ac8e63.
> > 2013-06-15 14:18:26,501 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001210077421002,1371213862480.3c695d05eaf2342e015149947dfccce2.
> > 2013-06-15 14:18:26,502 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001210077421002,1371215646288.bc0ae5fea3f27f66e87a6e6960e6ea22.
> > 2013-06-15 14:18:26,503 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001210854261002,1371215646288.c754faa28ce47cb12c72ecbbc1f7c85d.
> > 2013-06-15 14:18:26,504 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001211631141002,1371213862480.9ac7561a2297b5723c9bde2ac82a275b.
> > 2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001211631141002,1371215537292.c86caaeca975e84508593176d2403726.
> > 2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001220360631002,1371215537292.97757f6b0418dea7294af03a3fedfac1.
> > 2013-06-15 14:18:26,506 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001221137531002,1371211272356.79edc478ea385978cd954025dac8b9f9.
> > 2013-06-15 14:18:26,507 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001221137531002,1371213598407.9b1bbd5cd11099c5fedfd95bab0915df.
> > 2013-06-15 14:18:26,508 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001221137531002,1371215427123.357463550f00d07ffb25df0d84e0b837.
> > 2013-06-15 14:18:26,509 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001221914391002,1371215427123.71d31890d25ad0b6677a705cc7d71c37.
> > 2013-06-15 14:18:26,510 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001230648541002,1371213598407.e6bb294918d09dc08fd309807e208a2a.
> > 2013-06-15 14:18:26,511 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001230648541002,1371215306956.0c4e3c9a1cedcd40cca686a8b70bd980.
> > 2013-06-15 14:18:26,512 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001231425401002,1371215306956.283bada01f51766d0280219822c99a69.
> > 2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001280031821002,1371178911107.c0f3a84ffb0c39ed368941f6671d0cfd.
> > 2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001300080491002,1371178911107.27b811538f6ee109f267653f5f560b87.
> > 2013-06-15 14:18:26,515 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001330007771002,1371181427502.f6b2040f8a16b60107d71e32a810a371.
> > 2013-06-15 14:18:26,516 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001330007771002,1371207552053.add80b8254579a34fbbf070c0bb605bd.
> > 2013-06-15 14:18:26,517 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001331064311002,1371207552053.fe632f34b69579eccbd8b4e813b9088d.
> > 2013-06-15 14:18:26,518 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001340042061002,1371181427502.747aa41330a84f7d5fe696defc10f985.
> > 2013-06-15 14:18:26,519 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001380041871002,1371180369431.e390609ca6beb9b7baf8f2b4796f9dda.
> > 2013-06-15 14:18:26,520 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001390138251002,1371180369431.23d67b78df8fb952861b6d60073b643a.
> > 2013-06-15 14:18:26,521 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001390138251002,1371210229806.6f0cd39d615be35106aeec04c526fcda.
> > 2013-06-15 14:18:26,522 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001390138251002,1371212995795.4f9df026d5b9026f4282edcc4f54a1ef.
> > 2013-06-15 14:18:26,523 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001391127371002,1371212995795.a62a6b8d13925f98ce7b9243b19b3d25.
> > 2013-06-15 14:18:26,524 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001400098641002,1371210229806.55941435b45764299ec5b6563ecf5d4b.
> > 2013-06-15 14:18:26,525 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001400098641002,1371212823051.ce6a942475c3223c26544beedd04594d.
> > 2013-06-15 14:18:26,526 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001401087781002,1371212823051.73837e8ba4a0598fecf560cb4a14b5e4.
> > 2013-06-15 14:18:26,527 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001410067721002,1371259508101.57037bf83841725172996ef4d9c1a7ee.
> > 2013-06-15 14:18:26,528 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001420069141002,1371259508101.9c1caf332f6325b60a087467c2b48fa9.
> > 2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001420069141002,1371261843284.937d2e422a6130b82228443ac68afd02.
> > 2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001421136541002,1371261843284.961e6d73f48f108fb82bcacbeb488775.
> > 2013-06-15 14:18:26,530 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001430035471002,1371258634059.1b8e1a437453a9e4529179205c5ba625.
> > 2013-06-15 14:18:26,531 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001440056171002,1371258634059.4e2ec5bdd3a8fcd599b1edaa4243333f.
> > 2013-06-15 14:18:26,532 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001440056171002,1371262374869.0acf2c312d53e933d55d36d6e6558abc.
> > 2013-06-15 14:18:26,533 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001441098211002,1371262374869.15bc8b334ff9407679903be126d0805d.
> > 2013-06-15 14:18:26,537 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001490005171002,1371180009315.660270fd8baafce2f0607a2996fb81e9.
> > 2013-06-15 14:18:26,538 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001490005171002,1371207839796.804b59d6e59d8d9c2506057cafa1b50f.
> > 2013-06-15 14:18:26,539 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001490005171002,1371209785865.96ec8a1b777cf3e93eb5f7e173455698.
> > 2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001490736191002,1371209785865.caf571b498f5d747c9dc77fcd290c161.
> > 2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001491467231002,1371207839796.74ca8bcf079e8281a8d5403e5ac9b1f3.
> > 2013-06-15 14:18:26,541 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001491467231002,1371209683706.b10e30402bf0637b915947c640fd7ca4.
> > 2013-06-15 14:18:26,542 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001500716371002,1371209683706.899d40e11dc4e5930a4ad49ad14d831b.
> > 2013-06-15 14:18:26,543 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001510010721002,1371180009315.9fd04413854e814904405e6298ea4ba4.
> > 2013-06-15 14:18:26,545 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001600069801002,1371181237169.f21a35fac1906ace208dda777b010514.
> > 2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001600069801002,1371208329446.5e56a0c7fc4ccc83d56b09abd586794c.
> > 2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001600869361002,1371208329446.47ead382d396a937c1c25a18da6d7138.
> > 2013-06-15 14:18:26,547 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001610159121002,1371181237169.186c47cc02f5258d4f86699ece9b7823.
> > 2013-06-15 14:18:26,549 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001870007081002,1371259179922.8d20b1a06529454ec2d7f1f4247bf22e.
> > 2013-06-15 14:18:26,550 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001890716531002,1371259179922.4b096c2c516438baa3668ce235fc9c90.
> > 2013-06-15 14:18:26,551 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001890716531002,1371262229170.d40579d02a66226656a1e48e0aa5fef0.
> > 2013-06-15 14:18:26,552 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001890716531002,1371263273244.72b3118398132ecf0b630a3aeaa0c588.
> > 2013-06-15 14:18:26,553 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001900211511002,1371263273244.62bd89081bfbc0e1bfca7db214eebc8c.
> > 2013-06-15 14:18:26,554 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001901130951002,1371262229170.658e697026a46c7a1097213c2b7dbc59.
> > 2013-06-15 14:18:26,555 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001901130951002,1371263138497.f9e6520a49c52462e1a888bc3e62f71a.
> > 2013-06-15 14:18:26,556 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001910587571002,1371263138497.3077fae3eeafdb11ae663bd7c6761418.
> > 2013-06-15 14:18:26,557 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001920003561002,1371177634606.6805f9fffb6384fc6befe8f4ef2c8478.
> > 2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001960038371002,1371177634606.61662ddd932e0e483adac0c0ad072d31.
> > 2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001960038371002,1371179042477.7ea6904a4aca54b84f006dcffa98353e.
> > 2013-06-15 14:18:26,559 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001960038371002,1371200342086.1f6ce38a8765a5de4dced1f616de590b.
> > 2013-06-15 14:18:26,560 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001960038371002,1371202134689.f6385d90b1d6c3dd0fad4a5207a632cc.
> > 2013-06-15 14:18:26,561 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001960038371002,1371203337958.f80a9a7182b7346c16b0e50565ead084.
> > 2013-06-15 14:18:26,562 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001960966991002,1371203337958.0969455703fbd4594fba90eca1a5b8a7.
> > 2013-06-15 14:18:26,563 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001961895651002,1371202134689.709bdc87c3f5cb60a0379bb55d9dc3f1.
> > 2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001961895651002,1371203142268.293fc7628be8c32235b68914562f0134.
> > 2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001962824291002,1371203142268.e6af9210314c5a4ac5e6f5a793c1b2f3.
> > 2013-06-15 14:18:26,565 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001970486031002,1371200342086.7ea869039a9d75c2d58a36bd1e3f43e2.
> > 2013-06-15 14:18:26,566 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001970486031002,1371201529596.c84ee1352bd7042f01c225a07ec9fda5.
> > 2013-06-15 14:18:26,567 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001970486031002,1371202941763.6c4f92536fb9eede8da7f9a01d431b80.
> > 2013-06-15 14:18:26,568 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001971414651002,1371202941763.969715998f0266c37e14e3ef89d3ea92.
> > 2013-06-15 14:18:26,569 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001972343311002,1371201529596.fcdc2b301fa6d7792dd3e5535e8cc58a.
> > 2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001972343311002,1371202732228.d091d6d827acc608724b10c5f2b72010.
> > 2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001973271951002,1371202732228.b7dc98ba29c04b199d9c839beadfb7e6.
> > 2013-06-15 14:18:26,571 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001980074991002,1371179042477.e8dd68c0f884a491e63f477cf2319164.
> > 2013-06-15 14:18:26,572 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001980074991002,1371209818804.b634bc05efa2d4b6eb98f85ad4fde93d.
> > 2013-06-15 14:18:26,573 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001980074991002,1371212403387.705ad001a08ff629c21c8038a7e87363.
> > 2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001980074991002,1371214663565.b13dd722151a2ea44a6aba5d7af1905b.
> > 2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001980074991002,1371216342982.25b9c7134d2baf28f85b7e588c42fa1f.
> > 2013-06-15 14:18:26,575 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001980690071002,1371216342982.8a3ece0302a7aad860a9ce98e418d664.
> > 2013-06-15 14:18:26,576 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001981305191002,1371214663565.7e7375671360e812eb6e7adeff39b3a2.
> > 2013-06-15 14:18:26,577 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001981305191002,1371216258668.69aa7ba14b3748dd78a035fe0fa73994.
> > 2013-06-15 14:18:26,578 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001981920291002,1371216258668.a9e3bd75ed74cffefe41b2622a489cda.
> > 2013-06-15 14:18:26,579 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001982535431002,1371212403387.3364f1a6b67a83de7f0e9b62064f6b48.
> > 2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001982535431002,1371214478503.a8e263e0ba0190e55da471542efb3fc9.
> > 2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001982535431002,1371216173963.a7a72feecebdd2c4d21be2808b4550a8.
> > 2013-06-15 14:18:26,581 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001990432891002,1371216173963.bf8da05c0e99900af0fbb1fc2c9a5b5a.
> > 2013-06-15 14:18:26,582 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001991048011002,1371214478503.ea2e8bc4b27b0a1b440586a197d67d7f.
> > 2013-06-15 14:18:26,583 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001991048011002,1371216090115.cbe8280d8b6ebbbb5386f1c34b1ab98b.
> > 2013-06-15 14:18:26,584 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001991663111002,1371216090115.bb548f5af5e0d9c2a34e456ff23472e1.
> > 2013-06-15 14:18:26,585 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001992278251002,1371209818804.a0e860fe725cc86184d8525d134d4344.
> > 2013-06-15 14:18:26,587 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001992278251002,1371211949705.6b119a297db81b2177909e51d833358e.
> > 2013-06-15 14:18:26,588 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001992278251002,1371214293670.f091e9463441e08f39e84074ab58f3f0.
> > 2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000001992278251002,1371215973835.1a1e1125671301df2b54f1a0527daa18.
> > 2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000002240013061002,1371215973835.42d223d52e9ecad03dd10ebe9542551e.
> > 2013-06-15 14:18:26,590 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000002620017131002,1371214293670.a462b626169d5286149303e76eb83f52.
> > 2013-06-15 14:18:26,591 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000002620017131002,1371215898321.aa059217f2f2788f3a23bad75ff34e26.
> > 2013-06-15 14:18:26,592 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000011000071171002,1371215898321.3a4c342e75751ec441a2d63c3f06a5f2.
> > 2013-06-15 14:18:26,593 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000011000686571002,1371211949705.3668041afa8e2cce5e8abb80a8d2de54.
> > 2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000011000686571002,1371214112540.9a285ce01761253f977dbed2def5b953.
> > 2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000011000686571002,1371215818016.2d2a6518a7bd35f0615fc990acec5238.
> > 2013-06-15 14:18:26,595 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000011001301951002,1371215818016.799bf01965ad79f97e128f6c80938809.
> > 2013-06-15 14:18:26,596 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000011001917351002,1371214112540.921f4f2af20f783f938223c49065b28e.
> > 2013-06-15 14:18:26,597 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000011001917351002,1371215735171.f8b8b79d2aa33182a60f16aafe449524.
> > 2013-06-15 14:18:26,598 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> > 2013-06-15 14:18:29,377 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:30,761 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:36,248 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:37,049 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:37,735 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:38,416 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:39,139 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:39,933 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:40,627 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:41,285 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:42,298 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:42,949 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:43,635 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:18:47,984 DEBUG
> > org.apache.hadoop.hbase.master.AssignmentManager: Handling
> > transition=RS_ZK_REGION_SPLITTING, server=dsjhd006,60020,1371197351220,
> > region=10fc985bae394d3de78d2c5595a7ee4b
> > 2013-06-15 14:18:48,220 DEBUG
> > org.apache.hadoop.hbase.master.AssignmentManager: Handling
> > transition=RS_ZK_REGION_SPLIT, server=dsjhd006,60020,1371197351220,
> > region=10fc985bae394d3de78d2c5595a7ee4b
> > 2013-06-15 14:18:48,221 DEBUG
> > org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handling SPLIT
> > event for 10fc985bae394d3de78d2c5595a7ee4b; deleting node
> > 2013-06-15 14:18:48,221 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> > master:60000-0x13f38ce639906c9 Deleting existing unassigned node for
> > 10fc985bae394d3de78d2c5595a7ee4b that is in expected state
> > RS_ZK_REGION_SPLIT
> > 2013-06-15 14:18:48,225 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> > master:60000-0x13f38ce639906c9 Successfully deleted unassigned node for
> > region 10fc985bae394d3de78d2c5595a7ee4b in expected state
> RS_ZK_REGION_SPLIT
> > 2013-06-15 14:18:48,226 INFO
> > org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handled SPLIT
> > event;
> >
> parent=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371276011390.10fc985bae394d3de78d2c5595a7ee4b.
> > daughter
> >
> a=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371277127973.060e12898838607dd55d6bb0de074af6.daughter
> >
> b=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5g>\xEC,1371277127973.0584b8152c974c7b22a618a9561f303e.
> > 2013-06-15 14:19:32,338 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> .META.
> > starting at key ''
> > 2013-06-15 14:19:32,338 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner
> to
> > startKey at ''
> > 2013-06-15 14:19:32,345 INFO org.apache.hadoop.hbase.master.LoadBalancer:
> > Skipping load balancing because balanced cluster; servers=6 regions=890
> > average=148.33333 mostloaded=168 leastloaded=135
> > 2013-06-15 14:19:32,401 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Finished with scanning at
> > {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED =>
> 1028785192,}
> > 2013-06-15 14:19:32,735 DEBUG
> > org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 983 catalog row(s)
> > and gc'd 0 unreferenced parent region(s)
> > 2013-06-15 14:21:20,204 WARN org.apache.hadoop.conf.Configuration:
> > fs.default.name is deprecated. Instead, use fs.defaultFS
> > 2013-06-15 14:21:20,211 INFO org.apache.zookeeper.ZooKeeper: Initiating
> > client connection,
> connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> > sessionTimeout=180000
> >
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > 2013-06-15 14:21:20,213 INFO org.apache.zookeeper.ClientCnxn: Opening
> > socket connection to server dsjhd002/132.228.248.12:2181. Will not
> > attempt to authenticate using SASL (unknown error)
> > 2013-06-15 14:21:20,213 INFO
> > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> > this process is 72221@dsjhd001
> > 2013-06-15 14:21:20,213 DEBUG
> > org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
> > org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> > 2013-06-15 14:21:20,214 INFO org.apache.zookeeper.ClientCnxn: Socket
> > connection established to dsjhd002/132.228.248.12:2181, initiating
> session
> > 2013-06-15 14:21:20,219 INFO org.apache.zookeeper.ClientCnxn: Session
> > establishment complete on server dsjhd002/132.228.248.12:2181, sessionid
> > = 0x23f38ce677905da, negotiated timeout = 40000
> > 2013-06-15 14:21:20,225 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > Scanning .META. starting at row= for max=2147483647 rows using
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > 2013-06-15 14:21:20,281 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> .META.
> > starting at key 'cdr,,'
> > 2013-06-15 14:21:20,281 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner
> to
> > startKey at 'cdr,,'
> > 2013-06-15 14:21:20,288 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over
> .META.
> > starting at key 'cdr,,'
> > 2013-06-15 14:21:20,288 DEBUG
> > org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner
> to
> > startKey at 'cdr,,'
> > 2013-06-15 14:21:20,384 DEBUG
> > org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
> > org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> > 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ZooKeeper: Session:
> > 0x23f38ce677905da closed
> > 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ClientCnxn: EventThread
> > shut down
> > 2013-06-15 14:21:20,388 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > Scanning .META. starting at row=cdr,,00000000000000 for max=2147483647
> rows
> > using
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> > 2013-06-15 14:21:20,393 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> > cdr,,1371219562338.f80cffbfd14f542ebf62b699ff16cd1d.
> > 2013-06-15 14:21:20,394 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> > cdr,,1371220461424.9a60de5a8b93cbb090f2ff638700c17f.
> > 2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> > cdr,,1371220844707.671b124985ca6c8ec3ec2460cdeb3a6e.
> > 2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF+\x15\x8F,1371220844707.b8e125158cbe648f65975b7a55a64928.
> > 2013-06-15 14:21:20,397 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFFn\xE9U,1371220461424.c2829cb530417f67beabb0e0452e72aa.
> > 2013-06-15 14:21:20,399 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xA0\x87\x07,1371219562338.400457e7a191aa448aebb416306d99a8.
> > 2013-06-15 14:21:20,401 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371219208991.c77fc0904715b9aa6924856bf9b822b1.
> > 2013-06-15 14:21:20,402 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371221923804.d5f97df0277012c82585365c653752e1.
> > 2013-06-15 14:21:20,404 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371266084527.e63be214991f76f051ea86ea9c1e3c92.
> > 2013-06-15 14:21:20,406 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371268929218.fcf39c8f746f710b2412370635e931b3.
> > 2013-06-15 14:21:20,407 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371269885965.25544f24893033cc60a1f4dd498fb4ec.
> > 2013-06-15 14:21:20,409 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371270770875.242bf8d20a3718b3398297c38396d2c6.
> > 2013-06-15 14:21:20,411 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371271313678.1b796f461937f79c2491742c3672f487.
> > 2013-06-15 14:21:20,412 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371272501819.9c2db71ef235a396af00a40c2c60a30c.
> > 2013-06-15 14:21:20,414 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274172249.b92c0cde8329b6d0ae9515661b1ffdd7.
> > 2013-06-15 14:21:20,416 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274465375.88a56b9df947ca156d2f12e680fbf8fc.
> > 2013-06-15 14:21:20,418 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> > blocking until region is in META:
> >
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371276503503.2931a51dba76afdd4d1b1b69470658f9.
> >
> >
> > On Sat, Jun 15, 2013 at 11:05 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Have you checked master / region server logs w.r.t. such region ?
> >>
> >> Cheers
> >>
> >> On Sat, Jun 15, 2013 at 8:00 AM, hua beatls <be...@gmail.com> wrote:
> >>
> >> > HI,
> >> > we use CDH 4.2.0, find that some of the regions stop spliting and
> >> > increase contiusuly under heavy load. some regions become much bigger
> >> than
> >> > the setting value(4G),reaches 46g.
> >> > any reason or bugs?
> >> >
> >> > Thansk!
> >> >
> >> > beatls
> >> >
> >>
> >
> >
>
Re: WHY REGIONS STOP SPLITTING
Posted by hua beatls <be...@gmail.com>.
below is the region server's log:
too many flush and compaction in these regionservers, below is excerpt
from one of the regionserver's log:
2013-06-15 23:17:30,719 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
free to archive original file.
2013-06-15 23:17:30,729 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
to:
hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374
2013-06-15 23:17:30,729 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed compaction of 3 file(s) in cf of
receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.
into 6389524ff5784370a7d5e3c5f4cacfec, size=541.1m; total size for store is
2.3g
2013-06-15 23:17:30,729 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
completed compaction:
regionName=receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.,
storeName=cf, fileCount=3, fileSize=541.1m, priority=8,
time=1726303500804089; duration=20sec
2013-06-15 23:17:30,729 DEBUG
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
CompactSplitThread status: compaction_queue=(39:1), split_queue=0
2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting compaction on cf in region
receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.Store:
Starting compaction of 3 file(s) in cf of
receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
into
tmpdir=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp,
seqid=332549270, totalSize=300.9m
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
keycount=1217, bloomtype=NONE, size=45.5m, encoding=NONE
2013-06-15 23:17:30,977 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
with permission=rwxrwxrwx
2013-06-15 23:17:30,988 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:17:30,988 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1:
CompoundBloomFilterWriter
2013-06-15 23:17:40,944 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1)
2013-06-15 23:17:40,955 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
to
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/a0739ed1108a4a0dbf5055169b9c2ce1
2013-06-15 23:17:40,977 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Removing store files after compaction...
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving compacted store files.
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Starting to archive files:[class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40]
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
moving files to the archive directory:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf
2013-06-15 23:17:40,985 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
2013-06-15 23:17:40,986 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
free to archive original file.
2013-06-15 23:17:40,994 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
to:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
2013-06-15 23:17:40,994 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50
2013-06-15 23:17:40,995 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
free to archive original file.
2013-06-15 23:17:41,002 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
to:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50
2013-06-15 23:17:41,002 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40
2013-06-15 23:17:41,003 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
free to archive original file.
2013-06-15 23:17:41,010 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
to:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40
2013-06-15 23:17:41,011 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed compaction of 3 file(s) in cf of
receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
into a0739ed1108a4a0dbf5055169b9c2ce1, size=300.9m; total size for store is
2.1g
2013-06-15 23:17:41,011 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
completed compaction:
regionName=receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.,
storeName=cf, fileCount=3, fileSize=300.9m, priority=7,
time=1726323253116901; duration=10sec
2013-06-15 23:17:41,011 DEBUG
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
CompactSplitThread status: compaction_queue=(39:0), split_queue=0
2013-06-15 23:17:49,526 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Flush requested on
receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
2013-06-15 23:17:49,527 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.,
current region memstore size 128.1m
2013-06-15 23:17:49,530 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.,
commencing wait for mvcc, flushsize=134284248
2013-06-15 23:17:49,530 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-06-15 23:17:49,536 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
with permission=rwxrwxrwx
2013-06-15 23:17:49,544 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:17:49,544 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976:
CompoundBloomFilterWriter
2013-06-15 23:17:53,011 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976)
2013-06-15 23:17:53,011 INFO org.apache.hadoop.hbase.regionserver.Store:
Flushed , sequenceid=332587567, memsize=128.1m, into tmp file
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
2013-06-15 23:17:53,021 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Renaming flushed file at
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/3ebe00cb10714c3caa097954b80ad976
to
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/3ebe00cb10714c3caa097954b80ad976
2013-06-15 23:17:53,037 INFO org.apache.hadoop.hbase.regionserver.Store:
Added
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/3ebe00cb10714c3caa097954b80ad976,
entries=3411, sequenceid=332587567, filesize=127.7m
2013-06-15 23:17:53,038 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~128.1m/134284248, currentsize=1.7m/1810928 for
region
receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
in 3511ms, sequenceid=332587567, compaction requested=true
2013-06-15 23:17:53,039 DEBUG org.apache.hadoop.hbase.regionserver.Store:
6dd8975cfa5d08bce38f4794e90e7135 - cf: Initiating minorcompaction
2013-06-15 23:17:53,073 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Skipped compaction of cf. Only 2 file(s) of size 255.3m have met
compaction criteria.
2013-06-15 23:17:53,073 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Not compacting
receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
because compaction request was cancelled
2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Flush requested on
receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
current region memstore size 128.1m
2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
commencing wait for mvcc, flushsize=134284248
2013-06-15 23:17:56,062 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-06-15 23:17:56,066 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
with permission=rwxrwxrwx
2013-06-15 23:17:56,077 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:17:56,077 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645:
CompoundBloomFilterWriter
2013-06-15 23:17:58,986 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Flush requested on
receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.
2013-06-15 23:17:59,826 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645)
2013-06-15 23:17:59,826 INFO org.apache.hadoop.hbase.regionserver.Store:
Flushed , sequenceid=332597789, memsize=128.1m, into tmp file
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
2013-06-15 23:17:59,836 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Renaming flushed file at
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/a394dc28ac4a451693a9bf7d51e0a645
to
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/a394dc28ac4a451693a9bf7d51e0a645
2013-06-15 23:17:59,850 INFO org.apache.hadoop.hbase.regionserver.Store:
Added
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/a394dc28ac4a451693a9bf7d51e0a645,
entries=3411, sequenceid=332597789, filesize=127.7m
2013-06-15 23:17:59,852 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~128.1m/134284248, currentsize=3.2m/3346280 for
region
receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
in 3790ms, sequenceid=332597789, compaction requested=false
2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.Store:
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5-hdfs://mycluster/hbase/receipt/63ee011a1e063dc6261cdeeecae0fbf5/cf/13b4fedf7b17494580796906e544f26d-bottom
is not splittable
2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.,
current region memstore size 128.9m
2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.,
commencing wait for mvcc, flushsize=135110976
2013-06-15 23:17:59,852 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-06-15 23:17:59,855 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
with permission=rwxrwxrwx
2013-06-15 23:17:59,864 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:17:59,864 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1:
CompoundBloomFilterWriter
2013-06-15 23:18:03,367 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1)
2013-06-15 23:18:03,367 INFO org.apache.hadoop.hbase.regionserver.Store:
Flushed , sequenceid=332604970, memsize=128.9m, into tmp file
hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
2013-06-15 23:18:03,378 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Renaming flushed file at
hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/.tmp/a412083a9ac242a58d43132bac7cbeb1
to
hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/cf/a412083a9ac242a58d43132bac7cbeb1
2013-06-15 23:18:03,395 INFO org.apache.hadoop.hbase.regionserver.Store:
Added
hdfs://mycluster/hbase/receipt/8594d51c2a18b9b4c870b25102c75356/cf/a412083a9ac242a58d43132bac7cbeb1,
entries=3432, sequenceid=332604970, filesize=128.4m
2013-06-15 23:18:03,398 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~128.9m/135110976, currentsize=1.3m/1377880 for
region
receipt,0000000000000000000021100378291002,1371307236470.8594d51c2a18b9b4c870b25102c75356.
in 3546ms, sequenceid=332604970, compaction requested=false
2013-06-15 23:18:12,373 DEBUG
org.apache.hadoop.hbase.regionserver.LogRoller: HLog roll requested
2013-06-15 23:18:12,400 DEBUG
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: using new
createWriter -- HADOOP-6840
2013-06-15 23:18:12,400 DEBUG
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter:
Path=hdfs://mycluster/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309492374,
syncFs=true, hflush=true, compression=false
2013-06-15 23:18:12,400 DEBUG
org.apache.hadoop.hbase.regionserver.wal.HLog: cleanupCurrentWriter
waiting for transactions to get synced total 18683011 synced till here
18683000
2013-06-15 23:18:12,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
Roll
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309446700,
entries=73574, filesize=255195585. for
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371309492374
2013-06-15 23:18:12,433 DEBUG
org.apache.hadoop.hbase.regionserver.wal.HLog: Found 6 hlogs to remove out
of total 34; oldest outstanding sequenceid is 330673801 from region
1c5829d92a29beb415bcd2077ac21a60
2013-06-15 23:18:12,433 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
moving old hlog file
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371307974037
whose highest sequenceid is 330267393 to
/hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371307974037
2013-06-15 23:18:12,449 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
moving old hlog file
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308018234
whose highest sequenceid is 330351557 to
/hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308018234
2013-06-15 23:18:12,458 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
moving old hlog file
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308079693
whose highest sequenceid is 330422025 to
/hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308079693
2013-06-15 23:18:12,474 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
moving old hlog file
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308122233
whose highest sequenceid is 330489689 to
/hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308122233
2013-06-15 23:18:12,482 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
moving old hlog file
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308164543
whose highest sequenceid is 330561043 to
/hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308164543
2013-06-15 23:18:12,491 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
moving old hlog file
/hbase/.logs/dsjhd008,60020,1371197352209/dsjhd008%2C60020%2C1371197352209.1371308205474
whose highest sequenceid is 330632938 to
/hbase/.oldlogs/dsjhd008%2C60020%2C1371197352209.1371308205474
2013-06-15 23:18:16,003 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Flush requested on
receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.
2013-06-15 23:18:16,004 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
current region memstore size 128.1m
2013-06-15 23:18:16,004 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
commencing wait for mvcc, flushsize=134284248
2013-06-15 23:18:16,004 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-06-15 23:18:16,007 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
with permission=rwxrwxrwx
2013-06-15 23:18:16,017 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:18:16,017 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd:
CompoundBloomFilterWriter
2013-06-15 23:18:19,724 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/aface1c7cb724327b93d5d1d2d0a3237)
2013-06-15 23:18:19,739 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/.tmp/aface1c7cb724327b93d5d1d2d0a3237
to
hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/aface1c7cb724327b93d5d1d2d0a3237
2013-06-15 23:18:19,760 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Removing store files after compaction...
2013-06-15 23:18:19,762 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving compacted store files.
2013-06-15 23:18:19,762 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Starting to archive files:[class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5]
2013-06-15 23:18:19,762 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
moving files to the archive directory:
hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf
2013-06-15 23:18:19,763 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,765 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
free to archive original file.
2013-06-15 23:18:19,773 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5,
to:
hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/13b4fedf7b17494580796906e544f26d.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,773 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,775 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
free to archive original file.
2013-06-15 23:18:19,786 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5,
to:
hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/f78b18be172c458d82aed10e8cebfd6f.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,786 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,787 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
free to archive original file.
2013-06-15 23:18:19,798 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5,
to:
hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/1f01023b2e424c6490c217c9b4a8eb0f.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,798 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,800 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
free to archive original file.
2013-06-15 23:18:19,811 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5,
to:
hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/3c74ede880e445c7bb5b4987fc57ec9c.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,811 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,812 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5,
free to archive original file.
2013-06-15 23:18:19,823 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5,
to:
hdfs://mycluster/hbase/.archive/receipt/7e465efcf9e38bb8c8805a1dba0526a8/cf/c2e31f54599346aca2fbb8f444de9ef1.63ee011a1e063dc6261cdeeecae0fbf5
2013-06-15 23:18:19,824 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed major compaction of 5 file(s) in cf of
receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.
into aface1c7cb724327b93d5d1d2d0a3237, size=2.3g; total size for store is
3.0g
2013-06-15 23:18:19,824 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
completed compaction:
regionName=receipt,0000000000000000000002790275061002,1371308620672.7e465efcf9e38bb8c8805a1dba0526a8.,
storeName=cf, fileCount=5, fileSize=4.0g, priority=7,
time=1725499789501847; duration=1mins, 34sec
2013-06-15 23:18:19,824 DEBUG
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
CompactSplitThread status: compaction_queue=(39:0), split_queue=0
2013-06-15 23:18:19,824 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting compaction on cf in region
receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
2013-06-15 23:18:19,825 INFO org.apache.hadoop.hbase.regionserver.Store:
Starting compaction of 3 file(s) in cf of
receipt,0000000000000000000021090094361002,1371307215380.6dd8975cfa5d08bce38f4794e90e7135.
into
tmpdir=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp,
seqid=331203769, totalSize=1.0g
2013-06-15 23:18:19,825 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/dd0c60580a2f42ffb2529378dc792736,
keycount=13795, bloomtype=NONE, size=516.3m, encoding=NONE
2013-06-15 23:18:19,825 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/5afd93c11371463db2a01157de686030,
keycount=10233, bloomtype=NONE, size=383.0m, encoding=NONE
2013-06-15 23:18:19,825 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/cf/376827ca88674cddabbb5fe5192e72b1,
keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
2013-06-15 23:18:19,918 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/1e63ab1190ef49ad92b2316a06c19643
with permission=rwxrwxrwx
2013-06-15 23:18:19,925 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:18:19,925 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/6dd8975cfa5d08bce38f4794e90e7135/.tmp/1e63ab1190ef49ad92b2316a06c19643:
CompoundBloomFilterWriter
2013-06-15 23:18:22,023 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd)
2013-06-15 23:18:22,026 INFO org.apache.hadoop.hbase.regionserver.Store:
Flushed , sequenceid=332629815, memsize=128.1m, into tmp file
hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
2013-06-15 23:18:22,072 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Renaming flushed file at
hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/.tmp/4334387118ec4cc9aac9d540819148dd
to
hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/4334387118ec4cc9aac9d540819148dd
2013-06-15 23:18:22,096 INFO org.apache.hadoop.hbase.regionserver.Store:
Added
hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/4334387118ec4cc9aac9d540819148dd,
entries=3411, sequenceid=332629815, filesize=127.7m
2013-06-15 23:18:22,106 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~128.1m/134284248, currentsize=4.7m/4881632 for
region
receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.
in 6102ms, sequenceid=332629815, compaction requested=true
2013-06-15 23:18:22,106 DEBUG org.apache.hadoop.hbase.regionserver.Store:
hdfs://mycluster/hbase/receipt/cfbb3b99de7188d847c4b9f436c8bdb4/cf/c5339a0b6f914b3ea20981f536a69de4.031ef47e5e5792c2e3e6ec3d54df4474-hdfs://mycluster/hbase/receipt/031ef47e5e5792c2e3e6ec3d54df4474/cf/c5339a0b6f914b3ea20981f536a69de4-bottom
is not splittable
2013-06-15 23:18:22,107 DEBUG org.apache.hadoop.hbase.regionserver.Store:
cfbb3b99de7188d847c4b9f436c8bdb4 - cf: Initiating minorcompaction
2013-06-15 23:18:22,107 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Large Compaction
requested:
regionName=receipt,0000000000000000000021000421861002,1371302961456.cfbb3b99de7188d847c4b9f436c8bdb4.,
storeName=cf, fileCount=4, fileSize=796.4m (413.4m, 127.7m, 127.7m,
127.7m), priority=7, time=1726375653135879; Because:
regionserver60020.cacheFlusher; compaction_queue=(39:0), split_queue=0
On Sat, Jun 15, 2013 at 11:18 PM, hua beatls <be...@gmail.com> wrote:
> hi, TED,
> below is part of log from master server:
> 2013-06-15 14:18:18,262 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> 2013-06-15 14:18:26,147 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:26,153 INFO org.apache.zookeeper.ZooKeeper: Initiating
> client connection, connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> sessionTimeout=180000
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:18:26,155 INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> this process is 72221@dsjhd001
> 2013-06-15 14:18:26,155 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> 2013-06-15 14:18:26,155 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server dsjhd003/132.228.248.13:2181. Will not
> attempt to authenticate using SASL (unknown error)
> 2013-06-15 14:18:26,157 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to dsjhd003/132.228.248.13:2181, initiating session
> 2013-06-15 14:18:26,168 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server dsjhd003/132.228.248.13:2181, sessionid
> = 0x33f38cfddfd0598, negotiated timeout = 40000
> 2013-06-15 14:18:26,172 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row= for max=2147483647 rows using
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:18:26,210 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over .META.
> starting at key 'receipt,,'
> 2013-06-15 14:18:26,210 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner to
> startKey at 'receipt,,'
> 2013-06-15 14:18:26,213 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over .META.
> starting at key 'receipt,,'
> 2013-06-15 14:18:26,213 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner to
> startKey at 'receipt,,'
> 2013-06-15 14:18:26,473 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
> 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ZooKeeper: Session:
> 0x33f38cfddfd0598 closed
> 2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ClientCnxn: EventThread
> shut down
> 2013-06-15 14:18:26,485 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row=receipt,,00000000000000 for max=2147483647
> rows using
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:18:26,491 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001110007251002,1371260257669.f0f8b131fb77efb81f16eb01e493953f.
> 2013-06-15 14:18:26,492 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001110007251002,1371261674696.a9d0635ea62ea83eb847e2dc49c1c3a3.
> 2013-06-15 14:18:26,493 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001111085351002,1371261674696.5f600129659614930a10215e81b551d7.
> 2013-06-15 14:18:26,494 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001112163491002,1371260257669.c0358d1d04b4080d32199262c835a1fa.
> 2013-06-15 14:18:26,495 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001130002641002,1371260606344.c6eb5c6d48c5a2b2ae7612a287273108.
> 2013-06-15 14:18:26,496 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001131034981002,1371260606344.221332a360f91bb525a39dfdbe47a8b3.
> 2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001132067341002,1371260431691.5469992ccbb963ef7d27f53c2e05c88d.
> 2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001141017741002,1371260431691.61fada90a01436cce4342baae8b1a341.
> 2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001190035581002,1371178883452.cf8f2ad83efefa816340cd920bc3bbf8.
> 2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001210077421002,1371178883452.d8de7e1c6e7568c76a5337e3c53162fc.
> 2013-06-15 14:18:26,500 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001210077421002,1371211272356.2b646d6d9433dc78315756d491ac8e63.
> 2013-06-15 14:18:26,501 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001210077421002,1371213862480.3c695d05eaf2342e015149947dfccce2.
> 2013-06-15 14:18:26,502 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001210077421002,1371215646288.bc0ae5fea3f27f66e87a6e6960e6ea22.
> 2013-06-15 14:18:26,503 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001210854261002,1371215646288.c754faa28ce47cb12c72ecbbc1f7c85d.
> 2013-06-15 14:18:26,504 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001211631141002,1371213862480.9ac7561a2297b5723c9bde2ac82a275b.
> 2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001211631141002,1371215537292.c86caaeca975e84508593176d2403726.
> 2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001220360631002,1371215537292.97757f6b0418dea7294af03a3fedfac1.
> 2013-06-15 14:18:26,506 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001221137531002,1371211272356.79edc478ea385978cd954025dac8b9f9.
> 2013-06-15 14:18:26,507 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001221137531002,1371213598407.9b1bbd5cd11099c5fedfd95bab0915df.
> 2013-06-15 14:18:26,508 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001221137531002,1371215427123.357463550f00d07ffb25df0d84e0b837.
> 2013-06-15 14:18:26,509 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001221914391002,1371215427123.71d31890d25ad0b6677a705cc7d71c37.
> 2013-06-15 14:18:26,510 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001230648541002,1371213598407.e6bb294918d09dc08fd309807e208a2a.
> 2013-06-15 14:18:26,511 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001230648541002,1371215306956.0c4e3c9a1cedcd40cca686a8b70bd980.
> 2013-06-15 14:18:26,512 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001231425401002,1371215306956.283bada01f51766d0280219822c99a69.
> 2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001280031821002,1371178911107.c0f3a84ffb0c39ed368941f6671d0cfd.
> 2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001300080491002,1371178911107.27b811538f6ee109f267653f5f560b87.
> 2013-06-15 14:18:26,515 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001330007771002,1371181427502.f6b2040f8a16b60107d71e32a810a371.
> 2013-06-15 14:18:26,516 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001330007771002,1371207552053.add80b8254579a34fbbf070c0bb605bd.
> 2013-06-15 14:18:26,517 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001331064311002,1371207552053.fe632f34b69579eccbd8b4e813b9088d.
> 2013-06-15 14:18:26,518 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001340042061002,1371181427502.747aa41330a84f7d5fe696defc10f985.
> 2013-06-15 14:18:26,519 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001380041871002,1371180369431.e390609ca6beb9b7baf8f2b4796f9dda.
> 2013-06-15 14:18:26,520 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001390138251002,1371180369431.23d67b78df8fb952861b6d60073b643a.
> 2013-06-15 14:18:26,521 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001390138251002,1371210229806.6f0cd39d615be35106aeec04c526fcda.
> 2013-06-15 14:18:26,522 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001390138251002,1371212995795.4f9df026d5b9026f4282edcc4f54a1ef.
> 2013-06-15 14:18:26,523 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001391127371002,1371212995795.a62a6b8d13925f98ce7b9243b19b3d25.
> 2013-06-15 14:18:26,524 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001400098641002,1371210229806.55941435b45764299ec5b6563ecf5d4b.
> 2013-06-15 14:18:26,525 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001400098641002,1371212823051.ce6a942475c3223c26544beedd04594d.
> 2013-06-15 14:18:26,526 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001401087781002,1371212823051.73837e8ba4a0598fecf560cb4a14b5e4.
> 2013-06-15 14:18:26,527 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001410067721002,1371259508101.57037bf83841725172996ef4d9c1a7ee.
> 2013-06-15 14:18:26,528 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001420069141002,1371259508101.9c1caf332f6325b60a087467c2b48fa9.
> 2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001420069141002,1371261843284.937d2e422a6130b82228443ac68afd02.
> 2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001421136541002,1371261843284.961e6d73f48f108fb82bcacbeb488775.
> 2013-06-15 14:18:26,530 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001430035471002,1371258634059.1b8e1a437453a9e4529179205c5ba625.
> 2013-06-15 14:18:26,531 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001440056171002,1371258634059.4e2ec5bdd3a8fcd599b1edaa4243333f.
> 2013-06-15 14:18:26,532 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001440056171002,1371262374869.0acf2c312d53e933d55d36d6e6558abc.
> 2013-06-15 14:18:26,533 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001441098211002,1371262374869.15bc8b334ff9407679903be126d0805d.
> 2013-06-15 14:18:26,537 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001490005171002,1371180009315.660270fd8baafce2f0607a2996fb81e9.
> 2013-06-15 14:18:26,538 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001490005171002,1371207839796.804b59d6e59d8d9c2506057cafa1b50f.
> 2013-06-15 14:18:26,539 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001490005171002,1371209785865.96ec8a1b777cf3e93eb5f7e173455698.
> 2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001490736191002,1371209785865.caf571b498f5d747c9dc77fcd290c161.
> 2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001491467231002,1371207839796.74ca8bcf079e8281a8d5403e5ac9b1f3.
> 2013-06-15 14:18:26,541 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001491467231002,1371209683706.b10e30402bf0637b915947c640fd7ca4.
> 2013-06-15 14:18:26,542 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001500716371002,1371209683706.899d40e11dc4e5930a4ad49ad14d831b.
> 2013-06-15 14:18:26,543 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001510010721002,1371180009315.9fd04413854e814904405e6298ea4ba4.
> 2013-06-15 14:18:26,545 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001600069801002,1371181237169.f21a35fac1906ace208dda777b010514.
> 2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001600069801002,1371208329446.5e56a0c7fc4ccc83d56b09abd586794c.
> 2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001600869361002,1371208329446.47ead382d396a937c1c25a18da6d7138.
> 2013-06-15 14:18:26,547 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001610159121002,1371181237169.186c47cc02f5258d4f86699ece9b7823.
> 2013-06-15 14:18:26,549 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001870007081002,1371259179922.8d20b1a06529454ec2d7f1f4247bf22e.
> 2013-06-15 14:18:26,550 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001890716531002,1371259179922.4b096c2c516438baa3668ce235fc9c90.
> 2013-06-15 14:18:26,551 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001890716531002,1371262229170.d40579d02a66226656a1e48e0aa5fef0.
> 2013-06-15 14:18:26,552 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001890716531002,1371263273244.72b3118398132ecf0b630a3aeaa0c588.
> 2013-06-15 14:18:26,553 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001900211511002,1371263273244.62bd89081bfbc0e1bfca7db214eebc8c.
> 2013-06-15 14:18:26,554 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001901130951002,1371262229170.658e697026a46c7a1097213c2b7dbc59.
> 2013-06-15 14:18:26,555 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001901130951002,1371263138497.f9e6520a49c52462e1a888bc3e62f71a.
> 2013-06-15 14:18:26,556 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001910587571002,1371263138497.3077fae3eeafdb11ae663bd7c6761418.
> 2013-06-15 14:18:26,557 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001920003561002,1371177634606.6805f9fffb6384fc6befe8f4ef2c8478.
> 2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001960038371002,1371177634606.61662ddd932e0e483adac0c0ad072d31.
> 2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001960038371002,1371179042477.7ea6904a4aca54b84f006dcffa98353e.
> 2013-06-15 14:18:26,559 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001960038371002,1371200342086.1f6ce38a8765a5de4dced1f616de590b.
> 2013-06-15 14:18:26,560 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001960038371002,1371202134689.f6385d90b1d6c3dd0fad4a5207a632cc.
> 2013-06-15 14:18:26,561 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001960038371002,1371203337958.f80a9a7182b7346c16b0e50565ead084.
> 2013-06-15 14:18:26,562 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001960966991002,1371203337958.0969455703fbd4594fba90eca1a5b8a7.
> 2013-06-15 14:18:26,563 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001961895651002,1371202134689.709bdc87c3f5cb60a0379bb55d9dc3f1.
> 2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001961895651002,1371203142268.293fc7628be8c32235b68914562f0134.
> 2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001962824291002,1371203142268.e6af9210314c5a4ac5e6f5a793c1b2f3.
> 2013-06-15 14:18:26,565 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001970486031002,1371200342086.7ea869039a9d75c2d58a36bd1e3f43e2.
> 2013-06-15 14:18:26,566 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001970486031002,1371201529596.c84ee1352bd7042f01c225a07ec9fda5.
> 2013-06-15 14:18:26,567 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001970486031002,1371202941763.6c4f92536fb9eede8da7f9a01d431b80.
> 2013-06-15 14:18:26,568 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001971414651002,1371202941763.969715998f0266c37e14e3ef89d3ea92.
> 2013-06-15 14:18:26,569 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001972343311002,1371201529596.fcdc2b301fa6d7792dd3e5535e8cc58a.
> 2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001972343311002,1371202732228.d091d6d827acc608724b10c5f2b72010.
> 2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001973271951002,1371202732228.b7dc98ba29c04b199d9c839beadfb7e6.
> 2013-06-15 14:18:26,571 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001980074991002,1371179042477.e8dd68c0f884a491e63f477cf2319164.
> 2013-06-15 14:18:26,572 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001980074991002,1371209818804.b634bc05efa2d4b6eb98f85ad4fde93d.
> 2013-06-15 14:18:26,573 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001980074991002,1371212403387.705ad001a08ff629c21c8038a7e87363.
> 2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001980074991002,1371214663565.b13dd722151a2ea44a6aba5d7af1905b.
> 2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001980074991002,1371216342982.25b9c7134d2baf28f85b7e588c42fa1f.
> 2013-06-15 14:18:26,575 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001980690071002,1371216342982.8a3ece0302a7aad860a9ce98e418d664.
> 2013-06-15 14:18:26,576 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001981305191002,1371214663565.7e7375671360e812eb6e7adeff39b3a2.
> 2013-06-15 14:18:26,577 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001981305191002,1371216258668.69aa7ba14b3748dd78a035fe0fa73994.
> 2013-06-15 14:18:26,578 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001981920291002,1371216258668.a9e3bd75ed74cffefe41b2622a489cda.
> 2013-06-15 14:18:26,579 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001982535431002,1371212403387.3364f1a6b67a83de7f0e9b62064f6b48.
> 2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001982535431002,1371214478503.a8e263e0ba0190e55da471542efb3fc9.
> 2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001982535431002,1371216173963.a7a72feecebdd2c4d21be2808b4550a8.
> 2013-06-15 14:18:26,581 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001990432891002,1371216173963.bf8da05c0e99900af0fbb1fc2c9a5b5a.
> 2013-06-15 14:18:26,582 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001991048011002,1371214478503.ea2e8bc4b27b0a1b440586a197d67d7f.
> 2013-06-15 14:18:26,583 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001991048011002,1371216090115.cbe8280d8b6ebbbb5386f1c34b1ab98b.
> 2013-06-15 14:18:26,584 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001991663111002,1371216090115.bb548f5af5e0d9c2a34e456ff23472e1.
> 2013-06-15 14:18:26,585 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001992278251002,1371209818804.a0e860fe725cc86184d8525d134d4344.
> 2013-06-15 14:18:26,587 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001992278251002,1371211949705.6b119a297db81b2177909e51d833358e.
> 2013-06-15 14:18:26,588 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001992278251002,1371214293670.f091e9463441e08f39e84074ab58f3f0.
> 2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000001992278251002,1371215973835.1a1e1125671301df2b54f1a0527daa18.
> 2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000002240013061002,1371215973835.42d223d52e9ecad03dd10ebe9542551e.
> 2013-06-15 14:18:26,590 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000002620017131002,1371214293670.a462b626169d5286149303e76eb83f52.
> 2013-06-15 14:18:26,591 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000002620017131002,1371215898321.aa059217f2f2788f3a23bad75ff34e26.
> 2013-06-15 14:18:26,592 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000011000071171002,1371215898321.3a4c342e75751ec441a2d63c3f06a5f2.
> 2013-06-15 14:18:26,593 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000011000686571002,1371211949705.3668041afa8e2cce5e8abb80a8d2de54.
> 2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000011000686571002,1371214112540.9a285ce01761253f977dbed2def5b953.
> 2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000011000686571002,1371215818016.2d2a6518a7bd35f0615fc990acec5238.
> 2013-06-15 14:18:26,595 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000011001301951002,1371215818016.799bf01965ad79f97e128f6c80938809.
> 2013-06-15 14:18:26,596 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000011001917351002,1371214112540.921f4f2af20f783f938223c49065b28e.
> 2013-06-15 14:18:26,597 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000011001917351002,1371215735171.f8b8b79d2aa33182a60f16aafe449524.
> 2013-06-15 14:18:26,598 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
> 2013-06-15 14:18:29,377 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:30,761 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:36,248 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:37,049 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:37,735 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:38,416 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:39,139 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:39,933 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:40,627 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:41,285 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:42,298 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:42,949 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:43,635 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:18:47,984 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_SPLITTING, server=dsjhd006,60020,1371197351220,
> region=10fc985bae394d3de78d2c5595a7ee4b
> 2013-06-15 14:18:48,220 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_SPLIT, server=dsjhd006,60020,1371197351220,
> region=10fc985bae394d3de78d2c5595a7ee4b
> 2013-06-15 14:18:48,221 DEBUG
> org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handling SPLIT
> event for 10fc985bae394d3de78d2c5595a7ee4b; deleting node
> 2013-06-15 14:18:48,221 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:60000-0x13f38ce639906c9 Deleting existing unassigned node for
> 10fc985bae394d3de78d2c5595a7ee4b that is in expected state
> RS_ZK_REGION_SPLIT
> 2013-06-15 14:18:48,225 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:60000-0x13f38ce639906c9 Successfully deleted unassigned node for
> region 10fc985bae394d3de78d2c5595a7ee4b in expected state RS_ZK_REGION_SPLIT
> 2013-06-15 14:18:48,226 INFO
> org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handled SPLIT
> event;
> parent=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371276011390.10fc985bae394d3de78d2c5595a7ee4b.
> daughter
> a=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371277127973.060e12898838607dd55d6bb0de074af6.daughter
> b=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5g>\xEC,1371277127973.0584b8152c974c7b22a618a9561f303e.
> 2013-06-15 14:19:32,338 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over .META.
> starting at key ''
> 2013-06-15 14:19:32,338 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner to
> startKey at ''
> 2013-06-15 14:19:32,345 INFO org.apache.hadoop.hbase.master.LoadBalancer:
> Skipping load balancing because balanced cluster; servers=6 regions=890
> average=148.33333 mostloaded=168 leastloaded=135
> 2013-06-15 14:19:32,401 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Finished with scanning at
> {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192,}
> 2013-06-15 14:19:32,735 DEBUG
> org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 983 catalog row(s)
> and gc'd 0 unreferenced parent region(s)
> 2013-06-15 14:21:20,204 WARN org.apache.hadoop.conf.Configuration:
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2013-06-15 14:21:20,211 INFO org.apache.zookeeper.ZooKeeper: Initiating
> client connection, connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
> sessionTimeout=180000
> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:21:20,213 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server dsjhd002/132.228.248.12:2181. Will not
> attempt to authenticate using SASL (unknown error)
> 2013-06-15 14:21:20,213 INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> this process is 72221@dsjhd001
> 2013-06-15 14:21:20,213 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> 2013-06-15 14:21:20,214 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to dsjhd002/132.228.248.12:2181, initiating session
> 2013-06-15 14:21:20,219 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server dsjhd002/132.228.248.12:2181, sessionid
> = 0x23f38ce677905da, negotiated timeout = 40000
> 2013-06-15 14:21:20,225 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row= for max=2147483647 rows using
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:21:20,281 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over .META.
> starting at key 'cdr,,'
> 2013-06-15 14:21:20,281 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner to
> startKey at 'cdr,,'
> 2013-06-15 14:21:20,288 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Creating scanner over .META.
> starting at key 'cdr,,'
> 2013-06-15 14:21:20,288 DEBUG
> org.apache.hadoop.hbase.client.ClientScanner: Advancing internal scanner to
> startKey at 'cdr,,'
> 2013-06-15 14:21:20,384 DEBUG
> org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
> org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
> 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ZooKeeper: Session:
> 0x23f38ce677905da closed
> 2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ClientCnxn: EventThread
> shut down
> 2013-06-15 14:21:20,388 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> Scanning .META. starting at row=cdr,,00000000000000 for max=2147483647 rows
> using
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
> 2013-06-15 14:21:20,393 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,,1371219562338.f80cffbfd14f542ebf62b699ff16cd1d.
> 2013-06-15 14:21:20,394 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,,1371220461424.9a60de5a8b93cbb090f2ff638700c17f.
> 2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,,1371220844707.671b124985ca6c8ec3ec2460cdeb3a6e.
> 2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF+\x15\x8F,1371220844707.b8e125158cbe648f65975b7a55a64928.
> 2013-06-15 14:21:20,397 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFFn\xE9U,1371220461424.c2829cb530417f67beabb0e0452e72aa.
> 2013-06-15 14:21:20,399 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xA0\x87\x07,1371219562338.400457e7a191aa448aebb416306d99a8.
> 2013-06-15 14:21:20,401 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371219208991.c77fc0904715b9aa6924856bf9b822b1.
> 2013-06-15 14:21:20,402 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371221923804.d5f97df0277012c82585365c653752e1.
> 2013-06-15 14:21:20,404 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371266084527.e63be214991f76f051ea86ea9c1e3c92.
> 2013-06-15 14:21:20,406 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371268929218.fcf39c8f746f710b2412370635e931b3.
> 2013-06-15 14:21:20,407 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371269885965.25544f24893033cc60a1f4dd498fb4ec.
> 2013-06-15 14:21:20,409 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371270770875.242bf8d20a3718b3398297c38396d2c6.
> 2013-06-15 14:21:20,411 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371271313678.1b796f461937f79c2491742c3672f487.
> 2013-06-15 14:21:20,412 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371272501819.9c2db71ef235a396af00a40c2c60a30c.
> 2013-06-15 14:21:20,414 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274172249.b92c0cde8329b6d0ae9515661b1ffdd7.
> 2013-06-15 14:21:20,416 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274465375.88a56b9df947ca156d2f12e680fbf8fc.
> 2013-06-15 14:21:20,418 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
> blocking until region is in META:
> cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371276503503.2931a51dba76afdd4d1b1b69470658f9.
>
>
> On Sat, Jun 15, 2013 at 11:05 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Have you checked master / region server logs w.r.t. such region ?
>>
>> Cheers
>>
>> On Sat, Jun 15, 2013 at 8:00 AM, hua beatls <be...@gmail.com> wrote:
>>
>> > HI,
>> > we use CDH 4.2.0, find that some of the regions stop spliting and
>> > increase contiusuly under heavy load. some regions become much bigger
>> than
>> > the setting value(4G),reaches 46g.
>> > any reason or bugs?
>> >
>> > Thansk!
>> >
>> > beatls
>> >
>>
>
>
Re: WHY REGIONS STOP SPLITTING
Posted by hua beatls <be...@gmail.com>.
hi, TED,
below is part of log from master server:
2013-06-15 14:18:18,262 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
2013-06-15 14:18:26,147 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:26,153 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
sessionTimeout=180000
watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
2013-06-15 14:18:26,155 INFO
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
this process is 72221@dsjhd001
2013-06-15 14:18:26,155 DEBUG
org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
2013-06-15 14:18:26,155 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server dsjhd003/132.228.248.13:2181. Will not attempt
to authenticate using SASL (unknown error)
2013-06-15 14:18:26,157 INFO org.apache.zookeeper.ClientCnxn: Socket
connection established to dsjhd003/132.228.248.13:2181, initiating session
2013-06-15 14:18:26,168 INFO org.apache.zookeeper.ClientCnxn: Session
establishment complete on server dsjhd003/132.228.248.13:2181, sessionid =
0x33f38cfddfd0598, negotiated timeout = 40000
2013-06-15 14:18:26,172 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row= for max=2147483647 rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
2013-06-15 14:18:26,210 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'receipt,,'
2013-06-15 14:18:26,210 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'receipt,,'
2013-06-15 14:18:26,213 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'receipt,,'
2013-06-15 14:18:26,213 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'receipt,,'
2013-06-15 14:18:26,473 DEBUG
org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@3df9c11e
2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ZooKeeper: Session:
0x33f38cfddfd0598 closed
2013-06-15 14:18:26,484 INFO org.apache.zookeeper.ClientCnxn: EventThread
shut down
2013-06-15 14:18:26,485 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row=receipt,,00000000000000 for max=2147483647
rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
2013-06-15 14:18:26,491 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001110007251002,1371260257669.f0f8b131fb77efb81f16eb01e493953f.
2013-06-15 14:18:26,492 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001110007251002,1371261674696.a9d0635ea62ea83eb847e2dc49c1c3a3.
2013-06-15 14:18:26,493 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001111085351002,1371261674696.5f600129659614930a10215e81b551d7.
2013-06-15 14:18:26,494 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001112163491002,1371260257669.c0358d1d04b4080d32199262c835a1fa.
2013-06-15 14:18:26,495 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001130002641002,1371260606344.c6eb5c6d48c5a2b2ae7612a287273108.
2013-06-15 14:18:26,496 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001131034981002,1371260606344.221332a360f91bb525a39dfdbe47a8b3.
2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001132067341002,1371260431691.5469992ccbb963ef7d27f53c2e05c88d.
2013-06-15 14:18:26,497 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001141017741002,1371260431691.61fada90a01436cce4342baae8b1a341.
2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001190035581002,1371178883452.cf8f2ad83efefa816340cd920bc3bbf8.
2013-06-15 14:18:26,499 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001210077421002,1371178883452.d8de7e1c6e7568c76a5337e3c53162fc.
2013-06-15 14:18:26,500 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001210077421002,1371211272356.2b646d6d9433dc78315756d491ac8e63.
2013-06-15 14:18:26,501 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001210077421002,1371213862480.3c695d05eaf2342e015149947dfccce2.
2013-06-15 14:18:26,502 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001210077421002,1371215646288.bc0ae5fea3f27f66e87a6e6960e6ea22.
2013-06-15 14:18:26,503 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001210854261002,1371215646288.c754faa28ce47cb12c72ecbbc1f7c85d.
2013-06-15 14:18:26,504 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001211631141002,1371213862480.9ac7561a2297b5723c9bde2ac82a275b.
2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001211631141002,1371215537292.c86caaeca975e84508593176d2403726.
2013-06-15 14:18:26,505 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001220360631002,1371215537292.97757f6b0418dea7294af03a3fedfac1.
2013-06-15 14:18:26,506 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001221137531002,1371211272356.79edc478ea385978cd954025dac8b9f9.
2013-06-15 14:18:26,507 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001221137531002,1371213598407.9b1bbd5cd11099c5fedfd95bab0915df.
2013-06-15 14:18:26,508 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001221137531002,1371215427123.357463550f00d07ffb25df0d84e0b837.
2013-06-15 14:18:26,509 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001221914391002,1371215427123.71d31890d25ad0b6677a705cc7d71c37.
2013-06-15 14:18:26,510 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001230648541002,1371213598407.e6bb294918d09dc08fd309807e208a2a.
2013-06-15 14:18:26,511 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001230648541002,1371215306956.0c4e3c9a1cedcd40cca686a8b70bd980.
2013-06-15 14:18:26,512 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001231425401002,1371215306956.283bada01f51766d0280219822c99a69.
2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001280031821002,1371178911107.c0f3a84ffb0c39ed368941f6671d0cfd.
2013-06-15 14:18:26,514 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001300080491002,1371178911107.27b811538f6ee109f267653f5f560b87.
2013-06-15 14:18:26,515 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001330007771002,1371181427502.f6b2040f8a16b60107d71e32a810a371.
2013-06-15 14:18:26,516 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001330007771002,1371207552053.add80b8254579a34fbbf070c0bb605bd.
2013-06-15 14:18:26,517 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001331064311002,1371207552053.fe632f34b69579eccbd8b4e813b9088d.
2013-06-15 14:18:26,518 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001340042061002,1371181427502.747aa41330a84f7d5fe696defc10f985.
2013-06-15 14:18:26,519 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001380041871002,1371180369431.e390609ca6beb9b7baf8f2b4796f9dda.
2013-06-15 14:18:26,520 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001390138251002,1371180369431.23d67b78df8fb952861b6d60073b643a.
2013-06-15 14:18:26,521 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001390138251002,1371210229806.6f0cd39d615be35106aeec04c526fcda.
2013-06-15 14:18:26,522 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001390138251002,1371212995795.4f9df026d5b9026f4282edcc4f54a1ef.
2013-06-15 14:18:26,523 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001391127371002,1371212995795.a62a6b8d13925f98ce7b9243b19b3d25.
2013-06-15 14:18:26,524 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001400098641002,1371210229806.55941435b45764299ec5b6563ecf5d4b.
2013-06-15 14:18:26,525 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001400098641002,1371212823051.ce6a942475c3223c26544beedd04594d.
2013-06-15 14:18:26,526 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001401087781002,1371212823051.73837e8ba4a0598fecf560cb4a14b5e4.
2013-06-15 14:18:26,527 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001410067721002,1371259508101.57037bf83841725172996ef4d9c1a7ee.
2013-06-15 14:18:26,528 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001420069141002,1371259508101.9c1caf332f6325b60a087467c2b48fa9.
2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001420069141002,1371261843284.937d2e422a6130b82228443ac68afd02.
2013-06-15 14:18:26,529 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001421136541002,1371261843284.961e6d73f48f108fb82bcacbeb488775.
2013-06-15 14:18:26,530 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001430035471002,1371258634059.1b8e1a437453a9e4529179205c5ba625.
2013-06-15 14:18:26,531 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001440056171002,1371258634059.4e2ec5bdd3a8fcd599b1edaa4243333f.
2013-06-15 14:18:26,532 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001440056171002,1371262374869.0acf2c312d53e933d55d36d6e6558abc.
2013-06-15 14:18:26,533 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001441098211002,1371262374869.15bc8b334ff9407679903be126d0805d.
2013-06-15 14:18:26,537 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001490005171002,1371180009315.660270fd8baafce2f0607a2996fb81e9.
2013-06-15 14:18:26,538 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001490005171002,1371207839796.804b59d6e59d8d9c2506057cafa1b50f.
2013-06-15 14:18:26,539 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001490005171002,1371209785865.96ec8a1b777cf3e93eb5f7e173455698.
2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001490736191002,1371209785865.caf571b498f5d747c9dc77fcd290c161.
2013-06-15 14:18:26,540 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001491467231002,1371207839796.74ca8bcf079e8281a8d5403e5ac9b1f3.
2013-06-15 14:18:26,541 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001491467231002,1371209683706.b10e30402bf0637b915947c640fd7ca4.
2013-06-15 14:18:26,542 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001500716371002,1371209683706.899d40e11dc4e5930a4ad49ad14d831b.
2013-06-15 14:18:26,543 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001510010721002,1371180009315.9fd04413854e814904405e6298ea4ba4.
2013-06-15 14:18:26,545 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001600069801002,1371181237169.f21a35fac1906ace208dda777b010514.
2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001600069801002,1371208329446.5e56a0c7fc4ccc83d56b09abd586794c.
2013-06-15 14:18:26,546 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001600869361002,1371208329446.47ead382d396a937c1c25a18da6d7138.
2013-06-15 14:18:26,547 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001610159121002,1371181237169.186c47cc02f5258d4f86699ece9b7823.
2013-06-15 14:18:26,549 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001870007081002,1371259179922.8d20b1a06529454ec2d7f1f4247bf22e.
2013-06-15 14:18:26,550 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001890716531002,1371259179922.4b096c2c516438baa3668ce235fc9c90.
2013-06-15 14:18:26,551 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001890716531002,1371262229170.d40579d02a66226656a1e48e0aa5fef0.
2013-06-15 14:18:26,552 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001890716531002,1371263273244.72b3118398132ecf0b630a3aeaa0c588.
2013-06-15 14:18:26,553 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001900211511002,1371263273244.62bd89081bfbc0e1bfca7db214eebc8c.
2013-06-15 14:18:26,554 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001901130951002,1371262229170.658e697026a46c7a1097213c2b7dbc59.
2013-06-15 14:18:26,555 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001901130951002,1371263138497.f9e6520a49c52462e1a888bc3e62f71a.
2013-06-15 14:18:26,556 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001910587571002,1371263138497.3077fae3eeafdb11ae663bd7c6761418.
2013-06-15 14:18:26,557 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001920003561002,1371177634606.6805f9fffb6384fc6befe8f4ef2c8478.
2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001960038371002,1371177634606.61662ddd932e0e483adac0c0ad072d31.
2013-06-15 14:18:26,558 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001960038371002,1371179042477.7ea6904a4aca54b84f006dcffa98353e.
2013-06-15 14:18:26,559 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001960038371002,1371200342086.1f6ce38a8765a5de4dced1f616de590b.
2013-06-15 14:18:26,560 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001960038371002,1371202134689.f6385d90b1d6c3dd0fad4a5207a632cc.
2013-06-15 14:18:26,561 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001960038371002,1371203337958.f80a9a7182b7346c16b0e50565ead084.
2013-06-15 14:18:26,562 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001960966991002,1371203337958.0969455703fbd4594fba90eca1a5b8a7.
2013-06-15 14:18:26,563 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001961895651002,1371202134689.709bdc87c3f5cb60a0379bb55d9dc3f1.
2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001961895651002,1371203142268.293fc7628be8c32235b68914562f0134.
2013-06-15 14:18:26,564 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001962824291002,1371203142268.e6af9210314c5a4ac5e6f5a793c1b2f3.
2013-06-15 14:18:26,565 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001970486031002,1371200342086.7ea869039a9d75c2d58a36bd1e3f43e2.
2013-06-15 14:18:26,566 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001970486031002,1371201529596.c84ee1352bd7042f01c225a07ec9fda5.
2013-06-15 14:18:26,567 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001970486031002,1371202941763.6c4f92536fb9eede8da7f9a01d431b80.
2013-06-15 14:18:26,568 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001971414651002,1371202941763.969715998f0266c37e14e3ef89d3ea92.
2013-06-15 14:18:26,569 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001972343311002,1371201529596.fcdc2b301fa6d7792dd3e5535e8cc58a.
2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001972343311002,1371202732228.d091d6d827acc608724b10c5f2b72010.
2013-06-15 14:18:26,570 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001973271951002,1371202732228.b7dc98ba29c04b199d9c839beadfb7e6.
2013-06-15 14:18:26,571 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001980074991002,1371179042477.e8dd68c0f884a491e63f477cf2319164.
2013-06-15 14:18:26,572 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001980074991002,1371209818804.b634bc05efa2d4b6eb98f85ad4fde93d.
2013-06-15 14:18:26,573 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001980074991002,1371212403387.705ad001a08ff629c21c8038a7e87363.
2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001980074991002,1371214663565.b13dd722151a2ea44a6aba5d7af1905b.
2013-06-15 14:18:26,574 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001980074991002,1371216342982.25b9c7134d2baf28f85b7e588c42fa1f.
2013-06-15 14:18:26,575 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001980690071002,1371216342982.8a3ece0302a7aad860a9ce98e418d664.
2013-06-15 14:18:26,576 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001981305191002,1371214663565.7e7375671360e812eb6e7adeff39b3a2.
2013-06-15 14:18:26,577 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001981305191002,1371216258668.69aa7ba14b3748dd78a035fe0fa73994.
2013-06-15 14:18:26,578 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001981920291002,1371216258668.a9e3bd75ed74cffefe41b2622a489cda.
2013-06-15 14:18:26,579 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001982535431002,1371212403387.3364f1a6b67a83de7f0e9b62064f6b48.
2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001982535431002,1371214478503.a8e263e0ba0190e55da471542efb3fc9.
2013-06-15 14:18:26,580 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001982535431002,1371216173963.a7a72feecebdd2c4d21be2808b4550a8.
2013-06-15 14:18:26,581 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001990432891002,1371216173963.bf8da05c0e99900af0fbb1fc2c9a5b5a.
2013-06-15 14:18:26,582 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001991048011002,1371214478503.ea2e8bc4b27b0a1b440586a197d67d7f.
2013-06-15 14:18:26,583 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001991048011002,1371216090115.cbe8280d8b6ebbbb5386f1c34b1ab98b.
2013-06-15 14:18:26,584 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001991663111002,1371216090115.bb548f5af5e0d9c2a34e456ff23472e1.
2013-06-15 14:18:26,585 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001992278251002,1371209818804.a0e860fe725cc86184d8525d134d4344.
2013-06-15 14:18:26,587 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001992278251002,1371211949705.6b119a297db81b2177909e51d833358e.
2013-06-15 14:18:26,588 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001992278251002,1371214293670.f091e9463441e08f39e84074ab58f3f0.
2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000001992278251002,1371215973835.1a1e1125671301df2b54f1a0527daa18.
2013-06-15 14:18:26,589 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000002240013061002,1371215973835.42d223d52e9ecad03dd10ebe9542551e.
2013-06-15 14:18:26,590 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000002620017131002,1371214293670.a462b626169d5286149303e76eb83f52.
2013-06-15 14:18:26,591 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000002620017131002,1371215898321.aa059217f2f2788f3a23bad75ff34e26.
2013-06-15 14:18:26,592 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000011000071171002,1371215898321.3a4c342e75751ec441a2d63c3f06a5f2.
2013-06-15 14:18:26,593 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000011000686571002,1371211949705.3668041afa8e2cce5e8abb80a8d2de54.
2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000011000686571002,1371214112540.9a285ce01761253f977dbed2def5b953.
2013-06-15 14:18:26,594 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000011000686571002,1371215818016.2d2a6518a7bd35f0615fc990acec5238.
2013-06-15 14:18:26,595 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000011001301951002,1371215818016.799bf01965ad79f97e128f6c80938809.
2013-06-15 14:18:26,596 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000011001917351002,1371214112540.921f4f2af20f783f938223c49065b28e.
2013-06-15 14:18:26,597 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000011001917351002,1371215735171.f8b8b79d2aa33182a60f16aafe449524.
2013-06-15 14:18:26,598 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
receipt,0000000000000000000021310017011002,1371215735171.8de35f70d4d33d8c2a89daaae3a52bcd.
2013-06-15 14:18:29,377 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:30,761 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:36,248 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:37,049 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:37,735 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:38,416 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:39,139 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:39,933 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:40,627 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:41,285 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:42,298 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:42,949 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:43,635 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:18:47,984 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_SPLITTING, server=dsjhd006,60020,1371197351220,
region=10fc985bae394d3de78d2c5595a7ee4b
2013-06-15 14:18:48,220 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_SPLIT, server=dsjhd006,60020,1371197351220,
region=10fc985bae394d3de78d2c5595a7ee4b
2013-06-15 14:18:48,221 DEBUG
org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handling SPLIT
event for 10fc985bae394d3de78d2c5595a7ee4b; deleting node
2013-06-15 14:18:48,221 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:60000-0x13f38ce639906c9 Deleting existing unassigned node for
10fc985bae394d3de78d2c5595a7ee4b that is in expected state
RS_ZK_REGION_SPLIT
2013-06-15 14:18:48,225 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:60000-0x13f38ce639906c9 Successfully deleted unassigned node for
region 10fc985bae394d3de78d2c5595a7ee4b in expected state RS_ZK_REGION_SPLIT
2013-06-15 14:18:48,226 INFO
org.apache.hadoop.hbase.master.handler.SplitRegionHandler: Handled SPLIT
event;
parent=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371276011390.10fc985bae394d3de78d2c5595a7ee4b.
daughter
a=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5\x03L\x7F,1371277127973.060e12898838607dd55d6bb0de074af6.daughter
b=cdr,\x00\x00\x01?B\xF6A\x84\x7F\xFF\xFF\xFF\xF5g>\xEC,1371277127973.0584b8152c974c7b22a618a9561f303e.
2013-06-15 14:19:32,338 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key ''
2013-06-15 14:19:32,338 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at ''
2013-06-15 14:19:32,345 INFO org.apache.hadoop.hbase.master.LoadBalancer:
Skipping load balancing because balanced cluster; servers=6 regions=890
average=148.33333 mostloaded=168 leastloaded=135
2013-06-15 14:19:32,401 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Finished with scanning at {NAME => '.META.,,1', STARTKEY => '', ENDKEY =>
'', ENCODED => 1028785192,}
2013-06-15 14:19:32,735 DEBUG
org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 983 catalog row(s)
and gc'd 0 unreferenced parent region(s)
2013-06-15 14:21:20,204 WARN org.apache.hadoop.conf.Configuration:
fs.default.name is deprecated. Instead, use fs.defaultFS
2013-06-15 14:21:20,211 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, connectString=dsjhd002:2181,dsjhd001:2181,dsjhd003:2181
sessionTimeout=180000
watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
2013-06-15 14:21:20,213 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server dsjhd002/132.228.248.12:2181. Will not attempt
to authenticate using SASL (unknown error)
2013-06-15 14:21:20,213 INFO
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
this process is 72221@dsjhd001
2013-06-15 14:21:20,213 DEBUG
org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
2013-06-15 14:21:20,214 INFO org.apache.zookeeper.ClientCnxn: Socket
connection established to dsjhd002/132.228.248.12:2181, initiating session
2013-06-15 14:21:20,219 INFO org.apache.zookeeper.ClientCnxn: Session
establishment complete on server dsjhd002/132.228.248.12:2181, sessionid =
0x23f38ce677905da, negotiated timeout = 40000
2013-06-15 14:21:20,225 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row= for max=2147483647 rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
2013-06-15 14:21:20,281 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'cdr,,'
2013-06-15 14:21:20,281 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'cdr,,'
2013-06-15 14:21:20,288 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'cdr,,'
2013-06-15 14:21:20,288 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'cdr,,'
2013-06-15 14:21:20,384 DEBUG
org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@4dca8ed9
2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ZooKeeper: Session:
0x23f38ce677905da closed
2013-06-15 14:21:20,386 INFO org.apache.zookeeper.ClientCnxn: EventThread
shut down
2013-06-15 14:21:20,388 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row=cdr,,00000000000000 for max=2147483647 rows
using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@757d1e91
2013-06-15 14:21:20,393 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,,1371219562338.f80cffbfd14f542ebf62b699ff16cd1d.
2013-06-15 14:21:20,394 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,,1371220461424.9a60de5a8b93cbb090f2ff638700c17f.
2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,,1371220844707.671b124985ca6c8ec3ec2460cdeb3a6e.
2013-06-15 14:21:20,395 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF+\x15\x8F,1371220844707.b8e125158cbe648f65975b7a55a64928.
2013-06-15 14:21:20,397 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFFn\xE9U,1371220461424.c2829cb530417f67beabb0e0452e72aa.
2013-06-15 14:21:20,399 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xA0\x87\x07,1371219562338.400457e7a191aa448aebb416306d99a8.
2013-06-15 14:21:20,401 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371219208991.c77fc0904715b9aa6924856bf9b822b1.
2013-06-15 14:21:20,402 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371221923804.d5f97df0277012c82585365c653752e1.
2013-06-15 14:21:20,404 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371266084527.e63be214991f76f051ea86ea9c1e3c92.
2013-06-15 14:21:20,406 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371268929218.fcf39c8f746f710b2412370635e931b3.
2013-06-15 14:21:20,407 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371269885965.25544f24893033cc60a1f4dd498fb4ec.
2013-06-15 14:21:20,409 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371270770875.242bf8d20a3718b3398297c38396d2c6.
2013-06-15 14:21:20,411 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371271313678.1b796f461937f79c2491742c3672f487.
2013-06-15 14:21:20,412 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371272501819.9c2db71ef235a396af00a40c2c60a30c.
2013-06-15 14:21:20,414 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274172249.b92c0cde8329b6d0ae9515661b1ffdd7.
2013-06-15 14:21:20,416 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371274465375.88a56b9df947ca156d2f12e680fbf8fc.
2013-06-15 14:21:20,418 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
blocking until region is in META:
cdr,\x00\x00\x01?B\xF4\xC0\xE6\x7F\xFF\xFF\xFF\xFF\xFE\xE9i,1371276503503.2931a51dba76afdd4d1b1b69470658f9.
On Sat, Jun 15, 2013 at 11:05 PM, Ted Yu <yu...@gmail.com> wrote:
> Have you checked master / region server logs w.r.t. such region ?
>
> Cheers
>
> On Sat, Jun 15, 2013 at 8:00 AM, hua beatls <be...@gmail.com> wrote:
>
> > HI,
> > we use CDH 4.2.0, find that some of the regions stop spliting and
> > increase contiusuly under heavy load. some regions become much bigger
> than
> > the setting value(4G),reaches 46g.
> > any reason or bugs?
> >
> > Thansk!
> >
> > beatls
> >
>
Re: WHY REGIONS STOP SPLITTING
Posted by Ted Yu <yu...@gmail.com>.
Have you checked master / region server logs w.r.t. such region ?
Cheers
On Sat, Jun 15, 2013 at 8:00 AM, hua beatls <be...@gmail.com> wrote:
> HI,
> we use CDH 4.2.0, find that some of the regions stop spliting and
> increase contiusuly under heavy load. some regions become much bigger than
> the setting value(4G),reaches 46g.
> any reason or bugs?
>
> Thansk!
>
> beatls
>