[GitHub] [hive] ashutosh-bapat commented on a change in pull request #735: HIVE-21960 : Avoid running stats updater and partition management task on a replicated table.

[GitBox <gi...@apache.org>]

ashutosh-bapat commented on a change in pull request #735: HIVE-21960 : Avoid running stats updater and partition management task on a replicated table.
URL: https://github.com/apache/hive/pull/735#discussion_r308302947
 
 

 ##########
 File path: ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java
 ##########
 @@ -220,6 +221,16 @@ private void stopWorkers() {
     String skipParam = table.getParameters().get(SKIP_STATS_AUTOUPDATE_PROPERTY);
     if ("true".equalsIgnoreCase(skipParam)) return null;
 
+    // If the table is being replicated into,
+    // 1. the stats are also replicated from the source, so we don't need those to be calculated
+    //    on the target again
+    // 2. updating stats requires a writeId to be created. Hence writeIds on source and target
+    //    can get out of sync when stats are updated. That can cause consistency issues.
+    String replTrgtParam = table.getParameters().get(ReplConst.REPL_TARGET_PROPERTY);
+    if (replTrgtParam != null && !replTrgtParam.isEmpty()) {
+      return null;
 
 Review comment:
   Added a debug instead. There's already a debug "Processing table", so adding an info didn't look symmetric. On a replicated database, this would produce a lot of INFOs since all the tables in that database will be target of replication.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org