[GitHub] [hudi] boneanxs commented on a diff in pull request #8741: [HUDI-5394] Fix tests for RowCustomColumnsSortPartitioner

["boneanxs (via GitHub)" <gi...@apache.org>]

boneanxs commented on code in PR #8741:
URL: https://github.com/apache/hudi/pull/8741#discussion_r1197339547


##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/execution/bulkinsert/TestBulkInsertInternalPartitionerForRows.java:
##########
@@ -103,8 +103,7 @@ public void testBulkInsertInternalPartitioner(BulkInsertSortMode sortMode,
                                                 boolean isGloballySorted,
                                                 boolean isLocallySorted,
                                                 boolean populateMetaFields) {
-    Dataset<Row> records1 = generateTestRecords();
-    Dataset<Row> records2 = generateTestRecords();
+    Dataset<Row> records = generateTestRecords();

Review Comment:
   `testCustomColumnSortPartitionerWithRows` was copied from `testBulkInsertInternalPartitioner`. And I looked `org.apache.hudi.execution.bulkinsert.TestBulkInsertInternalPartitioner#testBulkInsertInternalPartitioner:177`,  it actually generates two records sets with different union times:
   
   ```java
   JavaRDD<HoodieRecord> records1 = generateTestRecordsForBulkInsert(jsc);
       JavaRDD<HoodieRecord> records2 = generateTripleTestRecordsForBulkInsert(jsc);
   ```
   
   So I think this should be a mistake, and I think union it twice should be enough(Here different union times for different partitions?)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org