[GitHub] [hudi] prashantwason commented on a change in pull request #3083: [HUDI-2016] Fixed bootstrap of Metadata Table when some actions are in progress.

[GitBox <gi...@apache.org>]

prashantwason commented on a change in pull request #3083:
URL: https://github.com/apache/hudi/pull/3083#discussion_r655608706



##########
File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/metadata/TestHoodieBackedMetadata.java
##########
@@ -120,46 +120,63 @@ public void testDefaultNoMetadataTable() throws Exception {
     assertThrows(TableNotFoundException.class, () -> HoodieTableMetaClient.builder().setConf(hadoopConf).setBasePath(metadataTableBasePath).build());
 
     // Metadata table is not created if disabled by config
+    String firstCommitTime = HoodieActiveTimeline.createNewInstantTime();
     try (SparkRDDWriteClient client = new SparkRDDWriteClient(engineContext, getWriteConfig(true, false))) {
-      client.startCommitWithTime("001");
-      client.insert(jsc.emptyRDD(), "001");
+      client.startCommitWithTime(firstCommitTime);
+      client.insert(jsc.parallelize(dataGen.generateInserts(firstCommitTime, 5)), firstCommitTime);

Review comment:
       In the bootstrap path this will do nothing. But in the normal path (post bootstrap), this path also syncs the metadata table.
   
   The metadata table sync is indepdendant of the dataset commits. So its possible that the dataset commit may have completed and metadata table sync failed (due to some error or crash) in postWrite. So calling the sync again in preWrite ensures we sync again.
   
   @vinothchandar Do you think the sync in preWrite can be removed as an optimization?  




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org