[GitHub] [hive] klcopp commented on a change in pull request #1256: HIVE-23853: CRUD based compaction also should update ACID file version metadata

[GitBox <gi...@apache.org>]

klcopp commented on a change in pull request #1256:
URL: https://github.com/apache/hive/pull/1256#discussion_r455104994



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFile.java
##########
@@ -161,6 +163,7 @@ private static synchronized MemoryManager getThreadLocalOrcLlapMemoryManager(fin
         LlapProxy.isDaemon()) {
         memory(getThreadLocalOrcLlapMemoryManager(conf));
       }
+      isCompaction = tableProperties != null && "true".equals(tableProperties.getProperty(AcidUtils.COMPACTOR_TABLE_PROPERTY));

Review comment:
       can use AcidUtils#isCompactionTable here

##########
File path: ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java
##########
@@ -25,9 +25,11 @@
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collections;
+import java.util.HashSet;

Review comment:
       unused import

##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -4206,6 +4206,10 @@ private static void copyFiles(final HiveConf conf, final FileSystem destFs,
             }
           }
           files = fileStatuses.toArray(new FileStatus[files.length]);
+
+          if (HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_WRITE_ACID_VERSION_FILE)) {
+            AcidUtils.OrcAcidVersion.writeVersionFile(destf, destFs);
+          }

Review comment:
       Why write the version file to the destination file during MoveTask, if we already wrote one in FSOperator and we move everything that was written in FSOp to the destination file?

##########
File path: ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java
##########
@@ -25,9 +25,11 @@
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collections;
+import java.util.HashSet;
 import java.util.LinkedList;
 import java.util.List;
 import java.util.Optional;
+import java.util.Set;

Review comment:
       unused import




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org