You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Tije Kusnadi <ti...@adskom.com> on 2017/11/29 03:38:59 UTC
Query in hive error!
*Diagnostic Messages for this Task:
Error: java.io.IOException: java.io.IOException: Cannot obtain block
length for LocatedBlock{BP-1781152160-172.16.0.170-1483968868061:blk_1076822902_3082215;
getBlockSize()=153218; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[172.16.0.175:50010
<http://172.16.0.175:50010>,DS-c03666a5-2f54-4085-b635-e36b25daedca,DISK],
DatanodeInfoWithStorage[172.16.0.109:50010
<http://172.16.0.109:50010>,DS-11410139-58aa-4160-9c89-2459884e7f51,DISK]]}
at org.apache.hadoop.hive.io
<http://org.apache.hadoop.hive.io>.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
at org.apache.hadoop.hive.io
<http://org.apache.hadoop.hive.io>.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
at org.apache.hadoop.hive.ql.io
<http://org.apache.hadoop.hive.ql.io>.HiveInputFormat.getRecordReader(HiveInputFormat.java:252)
at org.apache.hadoop.hive.ql.io
<http://org.apache.hadoop.hive.ql.io>.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:703)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.io.IOException: Cannot obtain block length for
LocatedBlock{BP-1781152160-172.16.0.170-1483968868061:blk_1076822902_3082215;
getBlockSize()=153218; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[172.16.0.175:50010
<http://172.16.0.175:50010>,DS-c03666a5-2f54-4085-b635-e36b25daedca,DISK],
DatanodeInfoWithStorage[172.16.0.109:50010
<http://172.16.0.109:50010>,DS-11410139-58aa-4160-9c89-2459884e7f51,DISK]]}
at org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:427)
at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:335)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:271)
at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:263)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1584)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:309)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:305)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:305)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:780)
at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:109)
at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
at org.apache.hadoop.hive.ql.io
<http://org.apache.hadoop.hive.ql.io>.HiveInputFormat.getRecordReader(HiveInputFormat.java:250)
... 9 more
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 8 Reduce: 3 Cumulative CPU: 29.62 sec HDFS
Read: 143256373 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 29 seconds 620 msec*
*i try running the command :*
*hdfs fsck -openforwrite -files -blocks -locations / | grep
blk_1076822902_3082215**
*the result is this:*
*
0. BP-1781152160-172.16.0.170-1483968868061:blk_1076822902_3082215{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1,
replicas=[ReplicaUnderConstruction[[DISK]DS-c03666a5-2f54-4085-b635-e36b25daedca:NORMAL:172.16.0.175:50010|RWR],
ReplicaUnderConstruction[[DISK]DS-11410139-58aa-4160-9c89-2459884e7f51:NORMAL:172.16.0.109:50010|RWR]]}
len=153218 MISSING!*
What can i do ? Looking forward for an answer!
Thanks