[GitHub] [hudi] danny0405 commented on a diff in pull request #8245: [HUDI-5944] Added the ability to fix partitiion missing in hudi synctool

["danny0405 (via GitHub)" <gi...@apache.org>]

danny0405 commented on code in PR #8245:
URL: https://github.com/apache/hudi/pull/8245#discussion_r1145683921


##########
hudi-sync/hudi-adb-sync/src/main/java/org/apache/hudi/sync/adb/AdbSyncTool.java:
##########
@@ -189,6 +190,13 @@ private void syncHoodieTable(String tableName, boolean useRealtimeInputFormat, b
     }
     LOG.info("Last commit time synced was found:{}", lastCommitTimeSynced.orElse("null"));
 
+    String currentInstantTs = config.getString(META_SYNC_CURRENT_INSTANT_TS);
+    if (lastCommitTimeSynced.isPresent() && !currentInstantTs.isEmpty()) {
+      // If occ is enabled, the commit time may be earlier than lastCommitTimeSynced.
+      // In this case, the metadata of the current instant cannot be synchronized, resulting in the loss of partition metadata.
+      lastCommitTimeSynced = currentInstantTs.compareTo(lastCommitTimeSynced.get()) < 0 ? Option.of(currentInstantTs) : lastCommitTimeSynced;
+      LOG.info("Reset lastCommitTimeSynced " + lastCommitTimeSynced.get());

Review Comment:
   Somehow we need to rememer the client sync time for each writer, or keep all the instant times that was synced in server, the timestamp comparison is tricky and likely to produce bugs.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org