[GitHub] [hudi] prashantwason commented on a change in pull request #2441: [HUDI 1308] [WIP] rfc15 perf prod testing

[GitBox <gi...@apache.org>]

prashantwason commented on a change in pull request #2441:
URL: https://github.com/apache/hudi/pull/2441#discussion_r557004070



##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java
##########
@@ -198,14 +200,20 @@ public boolean archiveIfRequired(HoodieEngineContext context) throws IOException
     // If metadata table is enabled, do not archive instants which are more recent that the latest synced
     // instant on the metadata table. This is required for metadata table sync.
     if (config.useFileListingMetadata()) {
-      Option<String> lastSyncedInstantTime = table.metadata().getSyncedInstantTime();
-      if (lastSyncedInstantTime.isPresent()) {
-        LOG.info("Limiting archiving of instants to last synced instant on metadata table at " + lastSyncedInstantTime.get());
-        instants = instants.filter(i -> HoodieTimeline.compareTimestamps(i.getTimestamp(), HoodieTimeline.LESSER_THAN,
-            lastSyncedInstantTime.get()));
-      } else {
-        LOG.info("Not archiving as there is no instants yet on the metadata table");
-        instants = Stream.empty();
+      try (HoodieTableMetadata tableMetadata = HoodieTableMetadata.create(table.getContext(), config.getBasePath(), "/tmp/",

Review comment:
       This is the spillableMapPath. Can be change to config.getSpillableMapBasePath().




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org