You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2018/01/24 23:21:00 UTC
[jira] [Created] (HIVE-18530) Replication should skip MM table (for
now)
Daniel Dai created HIVE-18530:
---------------------------------
Summary: Replication should skip MM table (for now)
Key: HIVE-18530
URL: https://issues.apache.org/jira/browse/HIVE-18530
Project: Hive
Issue Type: Bug
Components: repl
Reporter: Daniel Dai
Assignee: Daniel Dai
Currently replication cannot handle transactional table (including MM table) until HIVE-18320. HIVE-17504 skips table with transactional=true explicitly. HIVE-18352 changes the logic to use AcidUtils.isAcidTable for the same purpose. However, isAcidTable returns false for mm table, thus Hive still dump mm table during replication.
Here is an error message during dump mm table:
{code}
ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask. java.io.FileNotFoundException: Path is not a file: /apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000
at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
INFO : Completed executing command(queryId=hive_20180119203438_293813df-7630-47fa-bc30-5ef7cbb42842); Time taken: 1.119 seconds
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask. java.io.FileNotFoundException: Path is not a file: /apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000
at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) (state=08S01,code=1)
0: jdbc:hive2://ctr-e137-1514896590304-25219-> Closing: 0: jdbc:hive2://ctr-e137-1514896590304-25219-02-000005.hwx.site:2181,ctr-e137-1514896590304-25219-02-000012.hwx.site:2181,ctr-e137-1514896590304-25219-02-000009.hwx.site:2181,ctr-e137-1514896590304-25219-02-000004.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_HOST@EXAMPLE.COM
{code}
We shall switch to use AcidUtils.isTransactionalTable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)