You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Laurent Chavet (JIRA)" <ji...@apache.org> on 2011/04/21 19:44:06 UTC
[jira] [Resolved] (HBASE-3804) HBase gets stuck with "Regions in
transition timed out" for .META. and -ROOT-
[ https://issues.apache.org/jira/browse/HBASE-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laurent Chavet resolved HBASE-3804.
-----------------------------------
Resolution: Not A Problem
see comments
> HBase gets stuck with "Regions in transition timed out" for .META. and -ROOT-
> -----------------------------------------------------------------------------
>
> Key: HBASE-3804
> URL: https://issues.apache.org/jira/browse/HBASE-3804
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.2
> Environment: CentOs 5.4 x64 kernel 2.6.18-238.5.1.el5
> Single node cluster
> Reporter: Laurent Chavet
> Priority: Blocker
>
> After putting some significant load on the sever (load around of 8 for approximatively 50 minutes) the following starts showing in the logs and then hbase is completely stuck: won't recover without a restart (I copy a large chunk of the master logs before the problem in case it helps; the bottom part about ROOT and META repeats forever):
> 9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305484717
> 2011-04-20 06:19:51,228 INFO org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: Using syncFs -- HDFS-200
> 2011-04-20 06:19:51,228 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Creating writer path=hdfs://hqsnosq
> l01:9000/hbase/Property/69e1f895ca9d6824b409015a9f9c03a3/recovered.edits/0000000000000098012 region=69e1f895ca9d6824b409
> 015a9f9c03a3
> 2011-04-20 06:19:51,539 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Pushed=38 entries from hdfs://HQSNO
> SQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305484717
> 2011-04-20 06:19:51,540 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting hlog 42 of 43: hdfs://HQS
> NOSQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251, length=67522024
> 2011-04-20 06:19:51,540 INFO org.apache.hadoop.hbase.util.FSUtils: Recovering file hdfs://HQSNOSQL01:9000/hbase/.logs/HQ
> SNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:52,543 INFO org.apache.hadoop.hbase.util.FSUtils: Finished lease recover attempt for hdfs://HQSNOSQL01:
> 9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:53,336 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Pushed=40 entries from hdfs://HQSNO
> SQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:54,894 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting hlog 43 of 43: hdfs://HQS
> NOSQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805, length=75765855
> 2011-04-20 06:19:54,906 INFO org.apache.hadoop.hbase.util.FSUtils: Recovering file hdfs://HQSNOSQL01:9000/hbase/.logs/HQ
> SNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:55,108 INFO org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: Using syncFs -- HDFS-200
> 2011-04-20 06:19:55,108 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Creating writer path=hdfs://hqsnosq
> l01:9000/hbase/Property/a21bf162e9ff2915ff036f486919c9b5/recovered.edits/0000000000000098062 region=a21bf162e9ff2915ff03
> 6f486919c9b5
> 2011-04-20 06:19:55,908 INFO org.apache.hadoop.hbase.util.FSUtils: Finished lease recover attempt for hdfs://HQSNOSQL01:
> 9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:56,842 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Pushed=43 entries from hdfs://HQSNO
> SQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:56,859 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305143661 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305143661
> 2011-04-20 06:19:56,863 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305152287 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305152287
> 2011-04-20 06:19:56,867 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305160979 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305160979
> 2011-04-20 06:19:56,871 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305170430 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305170430
> 2011-04-20 06:19:56,875 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305178662 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305178662
> 2011-04-20 06:19:56,879 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305188551 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305188551
> 2011-04-20 06:19:56,883 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305196703 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305196703
> 2011-04-20 06:19:56,887 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305205039 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305205039
> 2011-04-20 06:19:56,891 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305213412 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305213412
> 2011-04-20 06:19:56,895 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305220665 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305220665
> 2011-04-20 06:19:56,899 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305231697 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305231697
> 2011-04-20 06:19:56,913 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305239174 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305239174
> 2011-04-20 06:19:56,917 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305248013 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305248013
> 2011-04-20 06:19:56,921 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305256411 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305256411
> 2011-04-20 06:19:56,925 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305265663 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305265663
> 2011-04-20 06:19:56,929 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305274667 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305274667
> 2011-04-20 06:19:56,933 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305283140 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305283140
> 2011-04-20 06:19:56,937 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305291307 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305291307
> 2011-04-20 06:19:56,941 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305300632 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305300632
> 2011-04-20 06:19:56,945 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305309439 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305309439
> 2011-04-20 06:19:56,949 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305316405 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305316405
> 2011-04-20 06:19:56,953 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305322730 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305322730
> 2011-04-20 06:19:56,957 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305330453 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305330453
> 2011-04-20 06:19:56,961 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305337020 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305337020
> 2011-04-20 06:19:56,965 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305345309 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305345309
> 2011-04-20 06:19:56,969 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305352766 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305352766
> 2011-04-20 06:19:56,972 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305362335 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305362335
> 2011-04-20 06:19:56,976 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305370474 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305370474
> 2011-04-20 06:19:56,980 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305379360 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305379360
> 2011-04-20 06:19:56,984 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305387434 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305387434
> 2011-04-20 06:19:56,988 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305396557 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305396557
> 2011-04-20 06:19:56,992 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305403848 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305403848
> 2011-04-20 06:19:56,996 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305412950 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305412950
> 2011-04-20 06:19:57,000 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305420349 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305420349
> 2011-04-20 06:19:57,004 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305429358 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305429358
> 2011-04-20 06:19:57,008 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305437383 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305437383
> 2011-04-20 06:19:57,012 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305446715 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305446715
> 2011-04-20 06:19:57,016 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305460169 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305460169
> 2011-04-20 06:19:57,678 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305467377 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305467377
> 2011-04-20 06:19:57,685 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305476873 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305476873
> 2011-04-20 06:19:57,689 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305484717 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305484717
> 2011-04-20 06:19:57,693 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:57,697 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805 to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:57,701 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Waiting for split writer threads to
> finish
> 2011-04-20 06:19:57,874 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Split writers finished
> 2011-04-20 06:19:57,881 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/36d89fc46915d8db56bad45c64a313a3/recovered.edits/0000000000000096979 (wrote 134 edits in 6107ms)
> 2011-04-20 06:19:57,885 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/4499b405bae1047884256f39b7559376/recovered.edits/0000000000000096571 (wrote 30 edits in 1338ms)
> 2011-04-20 06:19:57,889 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/465fb6b0a1f2d5306c7bb87145a32be2/recovered.edits/0000000000000096568 (wrote 22 edits in 909ms)
> 2011-04-20 06:19:57,893 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/4bc3c0bd3a824fb6c7fabbd332ef1009/recovered.edits/0000000000000097151 (wrote 85 edits in 3982ms)
> 2011-04-20 06:19:57,897 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/4bfff6ceb59cc4b6098eea42b49ee9ed/recovered.edits/0000000000000097336 (wrote 117 edits in 4810ms)
> 2011-04-20 06:19:57,901 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/52dee69cbe0890b54aee56bda54577c9/recovered.edits/0000000000000096567 (wrote 63 edits in 4445ms)
> 2011-04-20 06:19:57,905 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/539e4afe8064897395d4dc891605e702/recovered.edits/0000000000000097127 (wrote 53 edits in 3433ms)
> 2011-04-20 06:19:57,909 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/5728cb0bb67c7c3d8701fc0679db97b9/recovered.edits/0000000000000097473 (wrote 134 edits in 6623ms)
> 2011-04-20 06:19:57,913 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/5f92d0c1defc108140df27322257584d/recovered.edits/0000000000000096672 (wrote 77 edits in 7211ms)
> 2011-04-20 06:19:57,917 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/69e1f895ca9d6824b409015a9f9c03a3/recovered.edits/0000000000000098012 (wrote 10 edits in 306ms)
> 2011-04-20 06:19:57,921 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/70746a106ab152f290264d0b2b0f773c/recovered.edits/0000000000000096722 (wrote 117 edits in 7245ms)
> 2011-04-20 06:19:57,925 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/906e3283419ae67052c34e31cc40ff05/recovered.edits/0000000000000097951 (wrote 31 edits in 2389ms)
> 2011-04-20 06:19:57,929 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/94112f821b90271f9d693d5505d5eebf/recovered.edits/0000000000000097692 (wrote 71 edits in 4771ms)
> 2011-04-20 06:19:57,933 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/9f519dfd7370990d017e7f6d187968d3/recovered.edits/0000000000000096697 (wrote 77 edits in 5551ms)
> 2011-04-20 06:19:57,937 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/9f60e52d384d0fd9dfc2a2f3be42ac43/recovered.edits/0000000000000097877 (wrote 24 edits in 840ms)
> 2011-04-20 06:19:58,054 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/a21bf162e9ff2915ff036f486919c9b5/recovered.edits/0000000000000098062 (wrote 8 edits in 1860ms)
> 2011-04-20 06:19:58,061 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/abc26969d71b22de80c7e889cc9a6c12/recovered.edits/0000000000000096576 (wrote 5 edits in 281ms)
> 2011-04-20 06:19:58,065 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/b5d7c2bd8383ea8871890f40b9827dcf/recovered.edits/0000000000000097554 (wrote 125 edits in 5240ms)
> 2011-04-20 06:19:58,073 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/bf06b369cf84e5bccdd4e845636a2af6/recovered.edits/0000000000000096839 (wrote 1 edits in 17ms)
> 2011-04-20 06:19:58,077 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/cf46d330cdaa1120c8e1a5fcf1899b0f/recovered.edits/0000000000000097442 (wrote 1 edits in 850ms)
> 2011-04-20 06:19:58,081 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/d850f84bfdfa28c63934a0be5a41257e/recovered.edits/0000000000000096572 (wrote 53 edits in 2467ms)
> 2011-04-20 06:19:58,085 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/d9b94307130a7096032b44b3aa18d3ea/recovered.edits/0000000000000096795 (wrote 155 edits in 10534ms)
> 2011-04-20 06:19:58,088 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/da6540fdaae565a6d04cd78a145240dd/recovered.edits/0000000000000097580 (wrote 53 edits in 2992ms)
> 2011-04-20 06:19:58,092 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/df02c5d923c466c5b89337fecee920cf/recovered.edits/0000000000000096924 (wrote 1 edits in 25ms)
> 2011-04-20 06:19:58,096 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://hqsnosql01:9000/h
> base/Property/e90c1c020e3148756a57d977133e31fc/recovered.edits/0000000000000096569 (wrote 96 edits in 3862ms)
> 2011-04-20 06:19:58,097 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: hlog file splitting completed in 866
> 02 ms for hdfs://hqsnosql01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146
> 2011-04-20 06:19:58,102 INFO org.apache.hadoop.hbase.catalog.RootLocationEditor: Unsetting ROOT region location in ZooKe
> eper
> 2011-04-20 06:19:58,113 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12f72de7aee0000 Creating (or up
> dating) unassigned node for 70236052 with OFFLINE state
> 2011-04-20 06:19:58,133 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12f72de7aee0000 Creating (or up
> dating) unassigned node for 1028785192 with OFFLINE state
> 2011-04-20 06:19:58,139 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=M_ZK_REGION_OFFLINE,
> server=HQSNOSQL01:60000, region=1028785192/.META.
> 2011-04-20 06:20:36,363 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305598133
> 2011-04-20 06:20:36,364 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:20:36,364 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305598113
> 2011-04-20 06:20:36,364 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:20:36,364 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305598133
> 2011-04-20 06:20:36,364 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305598113
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:21:06,374 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:06,374 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305666374
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305666374
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:21:36,385 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305666374
> 2011-04-20 06:21:36,385 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305666374
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:22:06,393 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:06,393 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305726393
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305726394
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:22:36,402 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305726393
> 2011-04-20 06:22:36,402 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305726394
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:23:06,412 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:06,412 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305786412
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305786412
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:23:36,420 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305786412
> 2011-04-20 06:23:36,420 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305786412
> 2011-04-20 06:24:06,429 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305816420
> 2011-04-20 06:24:06,430 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:24:06,430 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305816420
> 2011-04-20 06:24:06,430 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:24:06,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=.META.,,1.102878519
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira