You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Mark Selby <ms...@pandora.com> on 2017/12/21 17:45:34 UTC
HBase ExportSnapshot and CorruptedSnapshotException: Can't find hfile
errors
We are using Cloudera CDH 5.12.1 on Ubuntu Xenial 16.04
hadoop 2.6.0+cdh5.12.1+2540-1.cdh5.12.1.p0.3~xenial-cdh5.12.1
hbase 1.2.0+cdh5.12.1+365-1.cdh5.12.1.p0.3~xenial-cdh5.12.1
We have 2 distinct Hadoop/HBase cluster which use Kerberos Authentication from the same realm
We take Nightly HBase snapshots of all our tables on our primary cluster and are attempting to use the ExportSnapshot utility to transfer our HBase data to our backup cluster for offline backups and a few other uses. This had been working fine for some time using an older version of CDH but since our upgrade to 5.12.1 and the securing of our cluster with Kerberos we have run into and issue that I just can not figure out.
We have 12 tables that we serial snapshot and attempt to export. If we have no snapshots at the remote site the export works just fine. Once we have a set of snapshots on remote side and attempt to send another set of snapshots the exports always fail with one of the snapshots are the Verify Snapshot Integrity step. The export always says that it can not find the hfile on the remote site even though the DistCP logs on the source side and the Namenode logs on the remote side say that the DistCP of the missing file actually did happen.
When I go to the remote site the hfile in question is indeed not there but I can figure out why. Any help in even further debugging would be greatly appreciated.
This is the command we use the export the HBase snapshots
HADOOP_OPTS="-Dmapred.job.map.memory.mb=4096 -Dmapreduce.map.memory.mb=4096 HADOOP_HEAPSIZE=4096 hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -chuser hbase -chgroup hbase -copy-to hdfs://hdnn001.ch1.tnbsound.com:8020/hbase -snapshot 20171207210001.idx_entity_dma
#############
# hbase log #
#############
2017-12-21 01:32:17,222 INFO [main] snapshot.ExportSnapshot: Copy Snapshot Manifest
2017-12-21 01:32:19,881 INFO [main] hdfs.DFSClient: Created token for distcp: HDFS_DELEGATION_TOKEN owner=distcp@TNBSOUND.COM, renewer=yarn, realUser=, issueDate=1513819939868, maxDate=1514424739868, sequenceNumber=189251, masterKeyId=245 on ha-hdfs:nbs-sv3-prod
2017-12-21 01:32:19,926 INFO [main] security.TokenCache: Got dt for hdfs://nbs-sv3-prod; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nbs-sv3-prod, Ident: (token for distcp: HDFS_DELEGATION_TOKEN owner=distcp@TNBSOUND.COM, renewer=yarn, realUser=, issueDate=1513819939868, maxDate=1514424739868, sequenceNumber=189251, masterKeyId=245)
2017-12-21 01:32:20,124 INFO [main] hdfs.DFSClient: Created token for distcp: HDFS_DELEGATION_TOKEN owner=distcp@TNBSOUND.COM, renewer=yarn, realUser=, issueDate=1513819940079, maxDate=1514424740079, sequenceNumber=454, masterKeyId=79 on 10.150.102.21:8020
2017-12-21 01:32:20,175 INFO [main] security.TokenCache: Got dt for hdfs://hdnn001.ch1.tnbsound.com:8020; Kind: HDFS_DELEGATION_TOKEN, Service: 10.150.102.21:8020, Ident: (token for distcp: HDFS_DELEGATION_TOKEN owner=distcp@TNBSOUND.COM, renewer=yarn, realUser=, issueDate=1513819940079, maxDate=1514424740079, sequenceNumber=454, masterKeyId=79)
2017-12-21 01:32:21,832 INFO [main] snapshot.ExportSnapshot: Loading Snapshot '20171220183027.idx_entity_asset_subregion' hfile list
2017-12-21 01:32:22,222 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2017-12-21 01:32:24,669 INFO [main] mapreduce.JobSubmitter: number of splits:1033
2017-12-21 01:32:25,122 INFO [main] mapreduce.JobSubmitter: Submitting tokens for job: job_1510510608409_35102
2017-12-21 01:32:25,123 INFO [main] mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nbs-sv3-prod, Ident: (token for distcp: HDFS_DELEGATION_TOKEN owner=distcp@TNBSOUND.COM, renewer=yarn, realUser=, issueDate=1513819939868, maxDate=1514424739868, sequenceNumber=189251, masterKeyId=245)
2017-12-21 01:32:25,124 INFO [main] mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: 10.150.102.21:8020, Ident: (token for distcp: HDFS_DELEGATION_TOKEN owner=distcp@TNBSOUND.COM, renewer=yarn, realUser=, issueDate=1513819940079, maxDate=1514424740079, sequenceNumber=454, masterKeyId=79)
2017-12-21 01:32:25,958 INFO [main] impl.YarnClientImpl: Submitted application application_1510510608409_35102
2017-12-21 01:32:26,018 INFO [main] mapreduce.Job: The url to track the job: http://hdrm001.sv3.tnbsound.com:8088/proxy/application_1510510608409_35102/
2017-12-21 01:32:26,018 INFO [main] mapreduce.Job: Running job: job_1510510608409_35102
2017-12-21 01:32:41,424 INFO [main] mapreduce.Job: Job job_1510510608409_35102 running in uber mode : false
2017-12-21 01:32:41,427 INFO [main] mapreduce.Job: map 0% reduce 0%
2017-12-21 01:33:32,243 INFO [main] mapreduce.Job: map 1% reduce 0%
2017-12-21 01:33:38,320 INFO [main] mapreduce.Job: map 2% reduce 0%
2017-12-21 01:33:43,381 INFO [main] mapreduce.Job: map 3% reduce 0%
2017-12-21 01:33:47,432 INFO [main] mapreduce.Job: map 4% reduce 0%
2017-12-21 01:33:55,529 INFO [main] mapreduce.Job: map 5% reduce 0%
2017-12-21 01:34:08,689 INFO [main] mapreduce.Job: map 6% reduce 0%
2017-12-21 01:34:39,048 INFO [main] mapreduce.Job: map 7% reduce 0%
2017-12-21 02:24:23,934 INFO [main] mapreduce.Job: map 8% reduce 0%
2017-12-21 02:30:55,419 INFO [main] mapreduce.Job: map 9% reduce 0%
2017-12-21 02:32:01,952 INFO [main] mapreduce.Job: map 10% reduce 0%
2017-12-21 02:32:34,208 INFO [main] mapreduce.Job: map 11% reduce 0%
2017-12-21 02:33:02,428 INFO [main] mapreduce.Job: map 12% reduce 0%
2017-12-21 02:33:31,662 INFO [main] mapreduce.Job: map 13% reduce 0%
2017-12-21 02:33:54,835 INFO [main] mapreduce.Job: map 14% reduce 0%
2017-12-21 02:34:14,995 INFO [main] mapreduce.Job: map 15% reduce 0%
2017-12-21 02:34:36,162 INFO [main] mapreduce.Job: map 16% reduce 0%
2017-12-21 02:34:52,282 INFO [main] mapreduce.Job: map 17% reduce 0%
2017-12-21 02:35:07,399 INFO [main] mapreduce.Job: map 18% reduce 0%
2017-12-21 02:35:21,510 INFO [main] mapreduce.Job: map 19% reduce 0%
2017-12-21 02:35:48,723 INFO [main] mapreduce.Job: map 20% reduce 0%
2017-12-21 02:36:10,891 INFO [main] mapreduce.Job: map 21% reduce 0%
2017-12-21 02:36:40,116 INFO [main] mapreduce.Job: map 22% reduce 0%
2017-12-21 02:37:11,353 INFO [main] mapreduce.Job: map 23% reduce 0%
2017-12-21 02:37:47,639 INFO [main] mapreduce.Job: map 24% reduce 0%
2017-12-21 02:38:20,886 INFO [main] mapreduce.Job: map 25% reduce 0%
2017-12-21 02:39:07,239 INFO [main] mapreduce.Job: map 26% reduce 0%
2017-12-21 02:40:37,987 INFO [main] mapreduce.Job: map 27% reduce 0%
2017-12-21 02:42:08,685 INFO [main] mapreduce.Job: map 28% reduce 0%
2017-12-21 02:44:22,727 INFO [main] mapreduce.Job: map 29% reduce 0%
2017-12-21 02:46:45,831 INFO [main] mapreduce.Job: map 30% reduce 0%
2017-12-21 02:49:30,100 INFO [main] mapreduce.Job: map 31% reduce 0%
2017-12-21 02:52:44,608 INFO [main] mapreduce.Job: map 32% reduce 0%
2017-12-21 02:54:07,240 INFO [main] mapreduce.Job: map 33% reduce 0%
2017-12-21 02:54:53,587 INFO [main] mapreduce.Job: map 34% reduce 0%
2017-12-21 02:55:29,855 INFO [main] mapreduce.Job: map 35% reduce 0%
2017-12-21 02:56:08,157 INFO [main] mapreduce.Job: map 36% reduce 0%
2017-12-21 02:56:52,519 INFO [main] mapreduce.Job: map 37% reduce 0%
2017-12-21 02:57:31,828 INFO [main] mapreduce.Job: map 38% reduce 0%
2017-12-21 02:58:43,346 INFO [main] mapreduce.Job: map 39% reduce 0%
2017-12-21 02:59:45,797 INFO [main] mapreduce.Job: map 40% reduce 0%
2017-12-21 03:01:17,452 INFO [main] mapreduce.Job: map 41% reduce 0%
2017-12-21 03:14:49,359 INFO [main] mapreduce.Job: map 42% reduce 0%
2017-12-21 03:23:24,007 INFO [main] mapreduce.Job: map 43% reduce 0%
2017-12-21 03:25:21,825 INFO [main] mapreduce.Job: map 44% reduce 0%
2017-12-21 03:26:10,184 INFO [main] mapreduce.Job: map 45% reduce 0%
2017-12-21 03:27:11,616 INFO [main] mapreduce.Job: map 46% reduce 0%
2017-12-21 03:28:13,066 INFO [main] mapreduce.Job: map 47% reduce 0%
2017-12-21 03:30:09,905 INFO [main] mapreduce.Job: map 48% reduce 0%
2017-12-21 03:32:11,752 INFO [main] mapreduce.Job: map 49% reduce 0%
2017-12-21 03:32:48,002 INFO [main] mapreduce.Job: map 50% reduce 0%
2017-12-21 03:33:13,181 INFO [main] mapreduce.Job: map 51% reduce 0%
2017-12-21 03:33:33,323 INFO [main] mapreduce.Job: map 52% reduce 0%
2017-12-21 03:33:50,440 INFO [main] mapreduce.Job: map 53% reduce 0%
2017-12-21 03:34:17,635 INFO [main] mapreduce.Job: map 54% reduce 0%
2017-12-21 03:34:40,794 INFO [main] mapreduce.Job: map 55% reduce 0%
2017-12-21 03:35:12,007 INFO [main] mapreduce.Job: map 56% reduce 0%
2017-12-21 03:35:43,218 INFO [main] mapreduce.Job: map 57% reduce 0%
2017-12-21 03:36:33,588 INFO [main] mapreduce.Job: map 58% reduce 0%
2017-12-21 03:37:44,080 INFO [main] mapreduce.Job: map 59% reduce 0%
2017-12-21 03:39:18,727 INFO [main] mapreduce.Job: map 60% reduce 0%
2017-12-21 03:40:50,343 INFO [main] mapreduce.Job: map 61% reduce 0%
2017-12-21 03:43:23,389 INFO [main] mapreduce.Job: map 62% reduce 0%
2017-12-21 03:46:54,847 INFO [main] mapreduce.Job: map 63% reduce 0%
2017-12-21 03:49:25,872 INFO [main] mapreduce.Job: map 64% reduce 0%
2017-12-21 03:50:28,300 INFO [main] mapreduce.Job: map 65% reduce 0%
2017-12-21 03:51:10,587 INFO [main] mapreduce.Job: map 66% reduce 0%
2017-12-21 03:52:00,928 INFO [main] mapreduce.Job: map 67% reduce 0%
2017-12-21 03:53:02,352 INFO [main] mapreduce.Job: map 68% reduce 0%
2017-12-21 03:54:31,956 INFO [main] mapreduce.Job: map 69% reduce 0%
2017-12-21 03:58:26,570 INFO [main] mapreduce.Job: map 70% reduce 0%
2017-12-21 04:07:53,345 INFO [main] mapreduce.Job: map 71% reduce 0%
2017-12-21 04:12:45,365 INFO [main] mapreduce.Job: map 72% reduce 0%
2017-12-21 04:15:48,617 INFO [main] mapreduce.Job: map 73% reduce 0%
2017-12-21 04:19:35,094 INFO [main] mapreduce.Job: map 74% reduce 0%
2017-12-21 04:20:32,468 INFO [main] mapreduce.Job: map 75% reduce 0%
2017-12-21 04:21:04,687 INFO [main] mapreduce.Job: map 76% reduce 0%
2017-12-21 04:21:31,870 INFO [main] mapreduce.Job: map 77% reduce 0%
2017-12-21 04:21:58,041 INFO [main] mapreduce.Job: map 78% reduce 0%
2017-12-21 04:22:36,288 INFO [main] mapreduce.Job: map 79% reduce 0%
2017-12-21 04:23:22,580 INFO [main] mapreduce.Job: map 80% reduce 0%
2017-12-21 04:24:18,926 INFO [main] mapreduce.Job: map 81% reduce 0%
2017-12-21 04:25:42,442 INFO [main] mapreduce.Job: map 82% reduce 0%
2017-12-21 04:27:22,050 INFO [main] mapreduce.Job: map 83% reduce 0%
2017-12-21 04:29:34,850 INFO [main] mapreduce.Job: map 84% reduce 0%
2017-12-21 04:31:24,517 INFO [main] mapreduce.Job: map 85% reduce 0%
2017-12-21 04:33:48,345 INFO [main] mapreduce.Job: map 86% reduce 0%
2017-12-21 04:38:55,186 INFO [main] mapreduce.Job: map 87% reduce 0%
2017-12-21 04:40:57,925 INFO [main] mapreduce.Job: map 88% reduce 0%
2017-12-21 04:42:27,456 INFO [main] mapreduce.Job: map 89% reduce 0%
2017-12-21 04:44:32,199 INFO [main] mapreduce.Job: map 90% reduce 0%
2017-12-21 05:04:02,009 INFO [main] mapreduce.Job: map 91% reduce 0%
2017-12-21 05:05:54,667 INFO [main] mapreduce.Job: map 92% reduce 0%
2017-12-21 05:06:50,010 INFO [main] mapreduce.Job: map 93% reduce 0%
2017-12-21 05:07:59,428 INFO [main] mapreduce.Job: map 94% reduce 0%
2017-12-21 05:09:27,937 INFO [main] mapreduce.Job: map 95% reduce 0%
2017-12-21 05:12:03,781 INFO [main] mapreduce.Job: map 96% reduce 0%
2017-12-21 05:17:41,527 INFO [main] mapreduce.Job: map 97% reduce 0%
2017-12-21 05:19:16,016 INFO [main] mapreduce.Job: map 98% reduce 0%
2017-12-21 05:21:33,672 INFO [main] mapreduce.Job: map 99% reduce 0%
2017-12-21 05:55:59,661 INFO [main] mapreduce.Job: map 100% reduce 0%
2017-12-21 06:04:07,765 INFO [main] mapreduce.Job: Job job_1510510608409_35102 completed successfully
2017-12-21 06:04:07,945 INFO [main] mapreduce.Job: Counters: 37
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=197491962
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=24222636380348
HDFS: Number of bytes written=24222635138639
HDFS: Number of read operations=79613
HDFS: Number of large read operations=0
HDFS: Number of write operations=36460
Job Counters
Launched map tasks=1033
Other local map tasks=1033
Total time spent by all maps in occupied slots (ms)=44930841520
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=11232710380
Total vcore-milliseconds taken by all map tasks=11232710380
Total megabyte-milliseconds taken by all map tasks=46009181716480
Map-Reduce Framework
Map input records=10321
Map output records=0
Input split bytes=1241709
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=7541424
CPU time spent (ms)=464618880
Physical memory (bytes) snapshot=1726486073344
Virtual memory (bytes) snapshot=4040094453760
Total committed heap usage (bytes)=1657532841984
org.apache.hadoop.hbase.snapshot.ExportSnapshot$Counter
BYTES_COPIED=24222635138639
BYTES_EXPECTED=24222635138639
BYTES_SKIPPED=7238216458675
COPY_FAILED=0
FILES_COPIED=9096
FILES_SKIPPED=1225
MISSING_FILES=0
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=0
2017-12-21 06:04:07,947 INFO [main] snapshot.ExportSnapshot: Finalize the Snapshot Export
2017-12-21 06:04:08,306 INFO [main] snapshot.ExportSnapshot: Verify snapshot integrity
2017-12-21 06:04:08,310 INFO [main] Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS
2017-12-21 06:05:04,333 ERROR [VerifySnapshot-pool1-t1] snapshot.SnapshotReferenceUtil: Can't find hfile: 5d3803fa1bd44fa7b035de9b6feb850a in the real (hdfs://hdnn001.ch1.tnbsound.com:8020/hbase/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a) or archive (hdfs://hdnn001.ch1.tnbsound.com:8020/hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a) directory for the primary table.
2017-12-21 06:05:04,338 ERROR [main] snapshot.ExportSnapshot: Snapshot export failed
org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Can't find hfile: 5d3803fa1bd44fa7b035de9b6feb850a in the real (hdfs://hdnn001.ch1.tnbsound.com:8020/hbase/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a) or archive (hdfs://hdnn001.ch1.tnbsound.com:8020/hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a) directory for the primary table.
at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.concurrentVisitReferencedFiles(SnapshotReferenceUtil.java:260)
at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.concurrentVisitReferencedFiles(SnapshotReferenceUtil.java:223)
at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.verifySnapshot(SnapshotReferenceUtil.java:200)
at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.verifySnapshot(SnapshotReferenceUtil.java:184)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.verifySnapshot(ExportSnapshot.java:836)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:1035)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:1094)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:1098)
##################
# DistCP Job log #
##################
2017-12-21 02:32:18,045 INFO [main] org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper: copy completed for input=hdfs://nbs-sv3-prod/hbase/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a output=hdfs://hdnn001.ch1.tnbsound.com:8020/hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a
#################
# Remote NN log #
#################
2017-12-21 01:33:27,044 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081287743_7547082{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 01:34:34,460 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081288768_7548107{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 01:35:43,206 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081289825_7549164{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 01:36:54,698 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081290906_7550245{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 01:38:10,098 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081292000_7551339{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 01:39:11,366 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081293010_7552349{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW]]}
2017-12-21 01:40:14,676 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081294044_7553383{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 01:41:30,488 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081295227_7554566{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW]]}
2017-12-21 01:42:28,317 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081296174_7555513{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW]]}
2017-12-21 01:43:32,910 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081297203_7556542{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW]]}
2017-12-21 01:44:33,405 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081298172_7557511{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 01:45:30,173 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081299071_7558410{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 01:46:29,146 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081299996_7559335{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 01:47:33,526 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081301040_7560379{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 01:48:38,988 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081302097_7561436{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW]]}
2017-12-21 01:49:34,469 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081302991_7562330{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW]]}
2017-12-21 01:50:41,856 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081304049_7563388{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW]]}
2017-12-21 01:51:50,611 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081305137_7564476{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW]]}
2017-12-21 01:52:50,414 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081306100_7565439{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 01:53:48,724 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081307033_7566372{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 01:54:54,428 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081308064_7567403{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW]]}
2017-12-21 01:55:52,592 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081309000_7568339{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW]]}
2017-12-21 01:56:58,108 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081310057_7569396{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW]]}
2017-12-21 01:58:06,573 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081311136_7570475{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW]]}
2017-12-21 01:59:18,049 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081312258_7571597{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 02:00:22,205 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081313252_7572591{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW]]}
2017-12-21 02:01:22,787 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081314197_7573536{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 02:02:22,669 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081315157_7574496{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:03:26,526 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081316186_7575525{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 02:04:27,558 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081317140_7576479{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:05:41,235 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081318318_7577657{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW]]}
2017-12-21 02:06:42,199 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081319272_7578611{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 02:07:43,063 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081320247_7579586{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:09:01,222 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081321490_7580829{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:10:11,515 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081322616_7581955{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 02:11:20,656 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081323715_7583054{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW]]}
2017-12-21 02:12:22,846 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081324693_7584032{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW]]}
2017-12-21 02:13:20,720 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081325624_7584963{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:14:15,917 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081326503_7585842{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW]]}
2017-12-21 02:15:16,383 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081327452_7586791{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW]]}
2017-12-21 02:16:30,275 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081328579_7587918{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:17:28,425 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081329518_7588857{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW]]}
2017-12-21 02:18:25,825 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081330467_7589806{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 02:19:29,356 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081331439_7590778{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW]]}
2017-12-21 02:20:35,138 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081332498_7591837{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW]]}
2017-12-21 02:21:41,196 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081333540_7592879{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:22:48,921 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081334636_7593975{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 02:23:59,981 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081335792_7595131{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW]]}
2017-12-21 02:25:06,088 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081336847_7596186{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW]]}
2017-12-21 02:26:07,927 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081337840_7597179{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 02:27:09,483 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081338810_7598149{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW]]}
2017-12-21 02:28:17,946 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081339909_7599248{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6a85517f-bf1b-40ea-a2ad-cced4f125ab0:NORMAL:10.150.102.25:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW]]}
2017-12-21 02:29:24,474 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081340967_7600306{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-9f22d5cf-18d5-45b6-b6e3-64eb2f66e643:NORMAL:10.150.102.30:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW]]}
2017-12-21 02:30:29,377 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081341989_7601328{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6bf25c60-fd32-456f-b48f-ff850fbfc85c:NORMAL:10.150.102.26:1004|RBW], ReplicaUnderConstruction[[DISK]DS-a7e66b02-3316-4210-b84e-bbcdd052f6d8:NORMAL:10.150.102.31:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10a01126-13f0-4685-9429-46a537bb3905:NORMAL:10.150.102.29:1004|RBW]]}
2017-12-21 02:31:45,751 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a. BP-129780439-10.150.102.21-1507160939281 blk_1081343204_7602543{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-cf065f38-a607-4448-b9e5-e4cb2904e2fc:NORMAL:10.150.102.27:1004|RBW], ReplicaUnderConstruction[[DISK]DS-10061503-a101-4a76-b08c-db03e4389ff1:NORMAL:10.150.102.32:1004|RBW], ReplicaUnderConstruction[[DISK]DS-5ca18f56-0ed3-4d10-9ee6-a085e293e1b2:NORMAL:10.150.102.28:1004|RBW]]}
2017-12-21 02:32:20,526 INFO org.apache.hadoop.hdfs.StateChange: DIR* completeFile: /hbase/archive/data/default/idx_entity_asset_subregion/35367cf285b59a1a66067fd168caf2b6/m/5d3803fa1bd44fa7b035de9b6feb850a is closed by DFSClient_attempt_1510510608409_35102_m_000568_0_542204195_1