[GitHub] [hive] kasakrisz commented on a diff in pull request #3420: HIVE-26375: Invalid materialized view after rebuild if source table was compacted

[GitBox <gi...@apache.org>]

kasakrisz commented on code in PR #3420:
URL: https://github.com/apache/hive/pull/3420#discussion_r926392298


##########
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestMaterializedViewRebuild.java:
##########
@@ -97,7 +91,7 @@ public void testWhenMajorCompactionThenIncrementalMVRebuildIsStillAvailable() th
     txnHandler.cleanTxnToWriteIdTable();
 
     List<String> result = execSelectAndDumpData("explain cbo alter materialized view " + MV1 + " rebuild", driver, "");
-    Assert.assertEquals(INCREMENTAL_REBUILD_PLAN, result);
+    Assert.assertEquals(FULL_REBUILD_PLAN, result);

Review Comment:
   We search for update/delete operations in the `COMPLETED_TXN_COMPONENTS` affected source tables at MV rebuild. Records are deleted from this table at compaction. So after compaction we can not confirm whether there were any deletes of any of the source tables any longer. It is relevant since executing an incremental rebuild plan which expects insert operations in all source table only in case there were deletes leads to data corruption in the refreshed view.
   
   The second rebuild can be an incremental since the first rebuild resets the source tables snapshot to a fresh one and txn data of operations done since that first rebuild still exists in `COMPLETED_TXN_COMPONENTS`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org