[GitHub] [hudi] xushiyan commented on a diff in pull request #5091: [HUDI-3453] Fix HoodieBackedTableMetadata concurrent reading issue

[GitBox <gi...@apache.org>]

xushiyan commented on code in PR #5091:
URL: https://github.com/apache/hudi/pull/5091#discussion_r928195385


##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedTableMetadata.java:
##########
@@ -94,6 +102,53 @@ public void testTableOperations() throws Exception {
     verifyBaseMetadataTable();
   }
 
+  @Test
+  public void testMultiReaderForHoodieBackedTableMetadata() throws Exception {
+    final int taskNumber = 100;
+    HoodieTableType tableType = HoodieTableType.COPY_ON_WRITE;
+    init(tableType);
+    testTable.doWriteOperation("000001", INSERT, emptyList(), asList("p1"), 1);
+    HoodieBackedTableMetadata tableMetadata = new HoodieBackedTableMetadata(context, writeConfig.getMetadataConfig(), writeConfig.getBasePath(), writeConfig.getSpillableMapBasePath(), false);
+    assertTrue(tableMetadata.enabled());
+    List<String> metadataPartitions = tableMetadata.getAllPartitionPaths();
+    String partition = metadataPartitions.get(0);
+    String finalPartition = basePath + "/" + partition;
+    ArrayList<String> duplicatedPartitions = new ArrayList<>(taskNumber);
+    for (int i = 0; i < taskNumber; i++) {
+      duplicatedPartitions.add(finalPartition);
+    }
+    ExecutorService executors = Executors.newFixedThreadPool(taskNumber);
+    AtomicBoolean flag = new AtomicBoolean(false);
+    AtomicInteger count = new AtomicInteger(0);
+    AtomicInteger filesNumber = new AtomicInteger(0);
+
+    for (String part : duplicatedPartitions) {
+      executors.submit(new Runnable() {
+        @Override
+        public void run() {
+          try {
+            count.incrementAndGet();
+            while (true) {
+              if (count.get() == taskNumber) {
+                break;
+              }
+            }
+            FileStatus[] files = tableMetadata.getAllFilesInPartition(new Path(part));
+            filesNumber.addAndGet(files.length);
+            LOG.warn(Arrays.toString(files) + " : " + files.length);

Review Comment:
   this test itself is hard to understand already. this message does not really help much as it just print out some debug info. so i would suggest remove it. A passing test should not print anything other than a success message.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org