[jira] [Commented] (HUDI-2077) Flaky test: TestHoodieDeltaStreamer

["Sagar Sumit (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17431004#comment-17431004 ] 

Sagar Sumit commented on HUDI-2077:
-----------------------------------

There's one more flaky test.
*TestHoodieDeltaStreamer.testAsyncClusteringServiceWithCompaction*

https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_apis/build/builds/2725/logs/61

I ran locally 50 times but it never failed. From the CI logs, the reason seems to be incompatible metadata schema due to a rename. At this point, I am not sure how the rename happened. The writer schema which is the current schema is the correct one according to `HoodieMetadataRecord`. For reference I'm attaching the writer and table schemas.

I am going to disable the scehma validation (anyway this is disabled by default, but it is hardcoded to true in `HoodieBackedTableMetadataWriter`).

> Flaky test: TestHoodieDeltaStreamer
> -----------------------------------
>
>                 Key: HUDI-2077
>                 URL: https://issues.apache.org/jira/browse/HUDI-2077
>             Project: Apache Hudi
>          Issue Type: Sub-task
>          Components: Testing
>            Reporter: Raymond Xu
>            Assignee: Raymond Xu
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: 28.txt
>
>
> {code:java}
>  [INFO] Results:8520[INFO] 8521[ERROR] Errors: 8522[ERROR]   TestHoodieDeltaStreamer.testUpsertsMORContinuousModeWithMultipleWriters:716->testUpsertsContinuousModeWithMultipleWriters:831->runJobsInParallel:940 » Execution{code}
>  Search "testUpsertsMORContinuousModeWithMultipleWriters" in the log file for details.
> {quote} 
> 1730667 [pool-1461-thread-1] WARN org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer - Got error : }}
>  org.apache.hudi.exception.HoodieIOException: Could not check if hdfs://localhost:43333/user/vsts/continuous_mor_mulitwriter is a valid table 
>  at org.apache.hudi.exception.TableNotFoundException.checkTableValidity(TableNotFoundException.java:59) 
>  at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:112) 
>  at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:73) 
>  at org.apache.hudi.common.table.HoodieTableMetaClient$Builder.build(HoodieTableMetaClient.java:606) 
>  at org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer$TestHelpers.assertAtleastNDeltaCommitsAfterCommit(TestHoodieDeltaStreamer.java:322) 
>  at org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer.lambda$runJobsInParallel$8(TestHoodieDeltaStreamer.java:906) 
>  at org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer$TestHelpers.lambda$waitTillCondition$0(TestHoodieDeltaStreamer.java:347) 
>  at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
>  at java.lang.Thread.run(Thread.java:748) 
>  {{Caused by: java.net.ConnectException: Call From fv-az238-328/10.1.0.24 to localhost:43333 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: [http://wiki.apache.org/hadoop/ConnectionRefused]
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)