You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/04/01 06:26:00 UTC
[jira] [Work logged] (HIVE-24909) Skip the repl events from getting
logged in notification log
[ https://issues.apache.org/jira/browse/HIVE-24909?focusedWorklogId=575336&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-575336 ]
ASF GitHub Bot logged work on HIVE-24909:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Apr/21 06:25
Start Date: 01/Apr/21 06:25
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on a change in pull request #2101:
URL: https://github.com/apache/hive/pull/2101#discussion_r605391893
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/ReplTxnTask.java
##########
@@ -106,6 +107,7 @@ public int execute() {
CommitTxnRequest commitTxnRequest = new CommitTxnRequest(txnId);
commitTxnRequest.setReplPolicy(work.getReplPolicy());
commitTxnRequest.setWriteEventInfos(work.getWriteEventInfos());
+ commitTxnRequest.setTxn_type(TxnType.REPL_CREATED);
Review comment:
We don't need this to be set for AbortTxns?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java
##########
@@ -227,6 +228,12 @@ public long openTxn(Context ctx, String user, TxnType txnType) throws LockExcept
return openTxn(ctx, user, txnType, 0);
}
+ @Override
+ public long openTxn(Context ctx, String user, TxnType txnType, String dbUnderReplication) throws LockException {
Review comment:
Can't we re-use replOpenTxn by enhancing that to take txnType?
##########
File path: standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AbortTxnRequest.java
##########
@@ -13,17 +13,24 @@
private static final org.apache.thrift.protocol.TField TXNID_FIELD_DESC = new org.apache.thrift.protocol.TField("txnid", org.apache.thrift.protocol.TType.I64, (short)1);
private static final org.apache.thrift.protocol.TField REPL_POLICY_FIELD_DESC = new org.apache.thrift.protocol.TField("replPolicy", org.apache.thrift.protocol.TType.STRING, (short)2);
+ private static final org.apache.thrift.protocol.TField TXN_TYPE_FIELD_DESC = new org.apache.thrift.protocol.TField("txn_type", org.apache.thrift.protocol.TType.I32, (short)3);
Review comment:
you require thrift code generation
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -669,8 +669,10 @@ public OpenTxnsResponse openTxns(OpenTxnRequest rqst) throws MetaException {
}
List<PreparedStatement> insertPreparedStmts = null;
TxnType txnType = rqst.isSetTxn_type() ? rqst.getTxn_type() : TxnType.DEFAULT;
+ boolean isReplReplayOperation = rqst.isSetReplPolicy() && txnType == TxnType.REPL_CREATED;
+ boolean isReplDumpOrLoadOperation = rqst.isSetReplPolicy() && txnType != TxnType.REPL_CREATED;
Review comment:
isReplDumpOrLoadOperation -> isHiveReplTxn ?
##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -669,8 +669,10 @@ public OpenTxnsResponse openTxns(OpenTxnRequest rqst) throws MetaException {
}
List<PreparedStatement> insertPreparedStmts = null;
TxnType txnType = rqst.isSetTxn_type() ? rqst.getTxn_type() : TxnType.DEFAULT;
+ boolean isReplReplayOperation = rqst.isSetReplPolicy() && txnType == TxnType.REPL_CREATED;
Review comment:
isReplayedReplTxn?
when we can have txnType == TxnType.REPL_CREATED but rqst.isSetReplPolicy() == false?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 575336)
Time Spent: 40m (was: 0.5h)
> Skip the repl events from getting logged in notification log
> ------------------------------------------------------------
>
> Key: HIVE-24909
> URL: https://issues.apache.org/jira/browse/HIVE-24909
> Project: Hive
> Issue Type: Bug
> Reporter: Haymant Mangla
> Assignee: Haymant Mangla
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently REPL dump events are logged and replicated as a part of replication policy. Whenever one replication cycle completed, we always have one transaction left open on the target corresponding to repl dump operation. This will never be caught up without manually dealing with the transaction on target cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)