[GitHub] [hudi] nsivabalan commented on a change in pull request #4180: [HUDI-2903] get table schema from the last commit with data written

[GitBox <gi...@apache.org>]

nsivabalan commented on a change in pull request #4180:
URL: https://github.com/apache/hudi/pull/4180#discussion_r782690404



##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
##########
@@ -241,6 +246,26 @@ private void deleteInstantFile(HoodieInstant instant) {
     return readDataFromPath(detailPath);
   }
 
+  /**
+   * Get the last instant with data written, and convert this to HoodieCommitMetadata
+   */
+  public Option<Pair<HoodieInstant, HoodieCommitMetadata>> getLastCommitMetadataToWrite() {

Review comment:
       nit: getLastCommitMetadataWithValidData

##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java
##########
@@ -114,7 +113,7 @@ private MessageType getTableParquetSchemaFromDataFile() throws Exception {
           if (lastDeltaCommit.isPresent()) {
             HoodieInstant lastDeltaInstant = lastDeltaCommit.get();
             // read from the log file wrote
-            commitMetadata = HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(lastDeltaInstant).get(),
+            HoodieCommitMetadata commitMetadata = HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(lastDeltaInstant).get(),

Review comment:
       there is a chance that last delta commit is empty and does not have valid schema. so, the fix we have done for COW above, might also be required here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org