[GitHub] [hudi] vinothchandar commented on a change in pull request #4243: [HUDI-2952] Fixing metadata table for non-partitioned dataset

[GitBox <gi...@apache.org>]

vinothchandar commented on a change in pull request #4243:
URL: https://github.com/apache/hudi/pull/4243#discussion_r826414362



##########
File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestTable.java
##########
@@ -660,8 +660,10 @@ public String getLogFileNameById(String fileId, int version) {
     return FileSystemTestUtils.listRecursive(fs, new Path(Paths.get(basePath, partitionPath).toString())).stream()
         .filter(entry -> {
           boolean toReturn = true;
+          String filePath = entry.getPath().toString();
           String fileName = entry.getPath().getName();
-          if (fileName.equals(HoodiePartitionMetadata.HOODIE_PARTITION_METAFILE)) {
+          if (fileName.equals(HoodiePartitionMetadata.HOODIE_PARTITION_METAFILE) || (!fileName.contains("log") && !fileName.contains("parquet"))

Review comment:
       same here

##########
File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtils.java
##########
@@ -367,7 +367,9 @@ public static long getTotalMarkerFileCount(String basePath, String partitionPath
     if (Files.notExists(basePath)) {
       return Collections.emptyList();
     }
-    return Files.list(basePath).filter(entry -> !entry.getFileName().toString().equals(HoodieTableMetaClient.METAFOLDER_NAME)).collect(Collectors.toList());
+    return Files.list(basePath).filter(entry -> (!entry.getFileName().toString().equals(HoodieTableMetaClient.METAFOLDER_NAME)
+        && !entry.getFileName().toString().contains("parquet") && !entry.getFileName().toString().contains("log"))

Review comment:
       no hardcoding of parquet. lets please replace with standard helpers or constants

##########
File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java
##########
@@ -282,20 +284,22 @@ private static void processRollbackMetadata(HoodieActiveTimeline metadataTableTi
     List<HoodieRecord> records = new LinkedList<>();
     int[] fileChangeCount = {0, 0}; // deletes, appends
 
-    partitionToDeletedFiles.forEach((partition, deletedFiles) -> {
+    partitionToDeletedFiles.forEach((partitionName, deletedFiles) -> {
       fileChangeCount[0] += deletedFiles.size();
+      final String partition = partitionName.equals(EMPTY_PARTITION_NAME) ? NON_PARTITIONED_NAME : partitionName;

Review comment:
       pull this check into lambda




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org