You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Zhaojing Yu (Jira)" <ji...@apache.org> on 2022/10/01 12:26:00 UTC
[jira] [Updated] (HUDI-3054) Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter
[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhaojing Yu updated HUDI-3054:
------------------------------
Fix Version/s: (was: 0.12.1)
> Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter
> -----------------------------------------------------------------------
>
> Key: HUDI-3054
> URL: https://issues.apache.org/jira/browse/HUDI-3054
> Project: Apache Hudi
> Issue Type: Test
> Components: Testing, tests-ci
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Blocker
> Labels: pull-request-available
>
> Ref: [https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_apis/build/builds/4428/logs/21]
>
> {code:java}
> 2021-12-17T11:39:57.1645757Z [INFO] Running org.apache.hudi.client.TestHoodieClientMultiWriter
> 2021-12-17T11:39:57.3453991Z 339506 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit8865530218583640556/dataset/.hoodie/metadata
> 2021-12-17T11:39:57.3984328Z 339559 [dispatcher-event-loop-5] WARN org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:39:57.5278608Z 339689 [dispatcher-event-loop-2] WARN org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:39:57.9783107Z 340139 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit8865530218583640556/dataset/.hoodie/metadata
> 2021-12-17T11:39:57.9927490Z 340154 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit8865530218583640556/dataset/.hoodie/metadata
> 2021-12-17T11:40:10.1428665Z 352304 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:40:10.9930023Z 353149 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:40:11.4294603Z 353590 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit3262960667280061850/dataset/.hoodie/metadata
> 2021-12-17T11:40:11.4763085Z 353637 [dispatcher-event-loop-5] WARN org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:40:11.6014876Z 353762 [dispatcher-event-loop-2] WARN org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:40:12.0892513Z 354250 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit3262960667280061850/dataset/.hoodie/metadata
> 2021-12-17T11:40:12.1061317Z 354267 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit3262960667280061850/dataset/.hoodie/metadata
> 2021-12-17T11:40:23.1499732Z 365311 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:40:24.1626167Z 366323 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit294667857867877904/dataset/.hoodie/metadata
> 2021-12-17T11:40:24.1945944Z 366355 [dispatcher-event-loop-5] WARN org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:40:24.3084730Z 366469 [dispatcher-event-loop-2] WARN org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:40:24.7350862Z 366896 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit294667857867877904/dataset/.hoodie/metadata
> 2021-12-17T11:40:24.7482727Z 366909 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit294667857867877904/dataset/.hoodie/metadata
> 2021-12-17T11:40:43.1530857Z 385314 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:40:44.0641298Z 386225 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:40:44.5065216Z 386667 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit4425942033067835021/dataset/.hoodie/metadata
> 2021-12-17T11:40:44.5408012Z 386702 [dispatcher-event-loop-5] WARN org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:40:44.6779567Z 386839 [dispatcher-event-loop-1] WARN org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:40:45.1410815Z 387301 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit4425942033067835021/dataset/.hoodie/metadata
> 2021-12-17T11:40:45.1543539Z 387315 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit4425942033067835021/dataset/.hoodie/metadata
> 2021-12-17T11:41:03.1384862Z 405299 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:41:04.0150577Z 406176 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit5137952068799441319/dataset/.hoodie/metadata
> 2021-12-17T11:41:04.0514934Z 406212 [dispatcher-event-loop-5] WARN org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:41:04.1677826Z 406329 [dispatcher-event-loop-2] WARN org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:41:04.6164436Z 406777 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit5137952068799441319/dataset/.hoodie/metadata
> 2021-12-17T11:41:04.6343307Z 406791 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit5137952068799441319/dataset/.hoodie/metadata
> 2021-12-17T11:41:13.0144546Z 415165 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:41:13.8545883Z 416010 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:41:14.2624248Z 416423 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit1733672335771853967/dataset/.hoodie/metadata
> 2021-12-17T11:41:14.2957800Z 416457 [dispatcher-event-loop-5] WARN org.apache.spark.scheduler.TaskSetManager - Stage 0 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:41:14.3989543Z 416560 [dispatcher-event-loop-3] WARN org.apache.spark.scheduler.TaskSetManager - Stage 1 contains a task of very large size (101 KB). The maximum recommended task size is 100 KB.
> 2021-12-17T11:41:14.7647136Z 416926 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit1733672335771853967/dataset/.hoodie/metadata
> 2021-12-17T11:41:14.7772128Z 416938 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit1733672335771853967/dataset/.hoodie/metadata
> 2021-12-17T11:41:22.6703021Z 424828 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:41:23.5771003Z 425738 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit4023714709478834450/dataset/.hoodie/metadata
> 2021-12-17T11:41:24.3341519Z 426495 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit4023714709478834450/dataset/.hoodie/metadata
> 2021-12-17T11:41:24.3488610Z 426510 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit4023714709478834450/dataset/.hoodie/metadata
> 2021-12-17T11:41:45.2669117Z 447428 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:41:46.1117672Z 448273 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:41:46.5285885Z 448689 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit6437731807948505371/dataset/.hoodie/metadata
> 2021-12-17T11:41:47.2040819Z 449365 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit6437731807948505371/dataset/.hoodie/metadata
> 2021-12-17T11:41:47.2190197Z 449380 [main] WARN org.apache.hudi.metadata.HoodieBackedTableMetadata - Metadata table was not found at path /tmp/junit6437731807948505371/dataset/.hoodie/metadata
> 2021-12-17T11:42:12.0807715Z 474241 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run
> 2021-12-17T11:42:12.8434589Z [ERROR] Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 135.656 s <<< FAILURE! - in org.apache.hudi.client.TestHoodieClientMultiWriter
> 2021-12-17T11:42:12.8435532Z [ERROR] org.apache.hudi.client.TestHoodieClientMultiWriter.testHoodieClientBasicMultiWriter(HoodieTableType)[1] Time elapsed: 13.652 s <<< ERROR!
> 2021-12-17T11:42:12.8436481Z java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.hudi.exception.HoodieWriteConflictException: java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes
> 2021-12-17T11:42:12.8437368Z at org.apache.hudi.client.TestHoodieClientMultiWriter.testHoodieClientBasicMultiWriter(TestHoodieClientMultiWriter.java:129)
> 2021-12-17T11:42:12.8438208Z Caused by: java.lang.RuntimeException: org.apache.hudi.exception.HoodieWriteConflictException: java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes
> 2021-12-17T11:42:12.8439052Z at org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testHoodieClientBasicMultiWriter$1(TestHoodieClientMultiWriter.java:125)
> 2021-12-17T11:42:12.8439840Z Caused by: org.apache.hudi.exception.HoodieWriteConflictException: java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes
> 2021-12-17T11:42:12.8440785Z at org.apache.hudi.client.TestHoodieClientMultiWriter.createCommitWithUpserts(TestHoodieClientMultiWriter.java:432)
> 2021-12-17T11:42:12.8441570Z at org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testHoodieClientBasicMultiWriter$1(TestHoodieClientMultiWriter.java:122)
> 2021-12-17T11:42:12.8442342Z Caused by: java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes
> 2021-12-17T11:42:12.8443007Z at org.apache.hudi.client.TestHoodieClientMultiWriter.createCommitWithUpserts(TestHoodieClientMultiWriter.java:432)
> 2021-12-17T11:42:12.8443792Z at org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testHoodieClientBasicMultiWriter$1(TestHoodieClientMultiWriter.java:122)
> 2021-12-17T11:42:12.8444188Z {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)