You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by 924602208 <do...@foxmail.com> on 2020/07/03 04:34:26 UTC
Hive job the block can't replicate

Hi,&nbsp;&nbsp; &nbsp;we have a hive job and the sink table is orc format, but it alway&nbsp;occur the exception when&nbsp;run the reducer phase occur the exception.


Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:210)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
&nbsp; &nbsp; &nbsp; &nbsp; at java.security.AccessController.doPrivileged(Native Method)
&nbsp; &nbsp; &nbsp; &nbsp; at javax.security.auth.Subject.doAs(Subject.java:422)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException(java.io.IOException):&nbsp;
File /tmp/staging/.hive-staging_hive_2020-07-03_12-23-06_538_3053216257266341950-1/_task_tmp.-ext-10002/_tmp.000000_3 could only be replicated to 0 nodes instead of minReplication (=1).&nbsp;
There are 9 datanode(s) running and no node(s) are excluded in this operation.
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1620)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3135)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3059)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:493)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
&nbsp; &nbsp; &nbsp;&nbsp;
when we change the orc format to the text format, it can be run succefully.


Best regards,
Eason