[GitHub] [iceberg] karuppayya commented on a change in pull request #2564: ICEBERG-2559: Add utility methods

[GitBox <gi...@apache.org>]

karuppayya commented on a change in pull request #2564:
URL: https://github.com/apache/iceberg/pull/2564#discussion_r629726033



##########
File path: core/src/main/java/org/apache/iceberg/MetadataLocationUtils.java
##########
@@ -58,17 +53,22 @@ private static void miscMetadataFiles(String metadataFileLocation, Set<String> m
     if (metadataFileLocation == null) {
       return;
     }
-    try {
-      TableMetadata metadata = TableMetadataParser.read(io, metadataFileLocation);
-      List<TableMetadata.MetadataLogEntry> metadataLogEntries = metadata.previousFiles();
-      List<String> previousMetadataFiles =
-          metadataLogEntries.stream().map(TableMetadata.MetadataLogEntry::file).collect(Collectors.toList());
+    TableMetadata metadata = TableMetadataParser.read(io, metadataFileLocation);
+    List<TableMetadata.MetadataLogEntry> metadataLogEntries = metadata.previousFiles();
+    List<String> previousMetadataFiles =
+        metadataLogEntries.stream().map(TableMetadata.MetadataLogEntry::file)
+            .collect(Collectors.toList());
+    if (previousMetadataFiles.size() > 0) {
       metaFiles.addAll(previousMetadataFiles);
-      if (isRecursive && previousMetadataFiles.size() > 0) {
-        miscMetadataFiles(previousMetadataFiles.get(0), metaFiles, io, isRecursive);
+      if (isRecursive) {

Review comment:
       In the above case, we will first pick the latest version(ie file 3), add all the files in file 3.
   Pick the oldest exist file from the list of files in file 3, and recurse with that file.
   ```
         if (isRecursive) {
           for (String fileName : previousMetadataFiles) {
             // Find the first existent metadata json file and recurse
             if (io.newInputFile(fileName).exists()) {
               miscMetadataFiles(fileName, metaFiles, io, isRecursive);
               break;
             }
           }
         }
   ```
   We dont queue up all the files present in a particular file. The max number of files queued up will be 1




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org