[GitHub] [hudi] vinothchandar commented on a change in pull request #4405: HUDI-3068 Fixing sync all partitions

[GitBox <gi...@apache.org>]

vinothchandar commented on a change in pull request #4405:
URL: https://github.com/apache/hudi/pull/4405#discussion_r773570338



##########
File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java
##########
@@ -123,6 +123,9 @@
   @Parameter(names = {"--conditional-sync"}, description = "If true, only sync on conditions like schema change or partition change.")
   public Boolean isConditionalSync = false;
 
+  @Parameter(names = {"--sync-all-partitions"}, description = "If true, sync all partitions")

Review comment:
       this is a force sync? if so may be make it clearer? `--force-sync-all-partitions` 

##########
File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/ddl/HMSDDLExecutor.java
##########
@@ -219,6 +219,7 @@ public void updatePartitionsToTable(String tableName, List<String> changedPartit
         sd.setLocation(fullPartitionPath);
         return new Partition(partitionValues, syncConfig.databaseName, tableName, 0, 0, sd, null);
       }).collect(Collectors.toList());
+      System.out.println("total partitions " +  partitionList.size());

Review comment:
       remove?

##########
File path: hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/AbstractSyncHoodieClient.java
##########
@@ -178,19 +178,27 @@ public boolean isDropPartition() {
     return false;
   }
 
-  @SuppressWarnings("OptionalUsedAsFieldOrParameterType")
   public List<String> getPartitionsWrittenToSince(Option<String> lastCommitTimeSynced) {
-    if (!lastCommitTimeSynced.isPresent()) {
+    return getPartitionsWrittenToSince(lastCommitTimeSynced,false);
+  }
+
+  @SuppressWarnings("OptionalUsedAsFieldOrParameterType")
+  public List<String> getPartitionsWrittenToSince(Option<String> lastCommitTimeSynced, boolean syncAllPartitions) {

Review comment:
       love to not overload this method with one more path. Can we branch off before in the call stack 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org