You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Davis Zhang (Jira)" <ji...@apache.org> on 2025/01/14 20:19:00 UTC
[jira] [Updated] (HUDI-8864) [Flaky test] org.apache.hudi.functional.TestStructuredStreaming
[ https://issues.apache.org/jira/browse/HUDI-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Davis Zhang updated HUDI-8864:
------------------------------
Parent: HUDI-8695
Issue Type: Sub-task (was: Task)
> [Flaky test] org.apache.hudi.functional.TestStructuredStreaming
> ---------------------------------------------------------------
>
> Key: HUDI-8864
> URL: https://issues.apache.org/jira/browse/HUDI-8864
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Davis Zhang
> Priority: Major
> Labels: flaky-test
> Fix For: 1.1.0
>
>
>
> org.apache.hudi.functional.TestStructuredStreaming
>
> at org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runBatch$16(MicroBatchExecution.scala:726) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4201|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4202] at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:411) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4202|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4203] at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:409) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4203|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4204] at org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:67) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4204|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4205] at org.apache.spark.sql.execution.streaming.MicroBatchExecution.runBatch(MicroBatchExecution.scala:726) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4205|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4206] at org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:284) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4206|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4207] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) ~[scala-library-2.12.17.jar:?]
> [4207|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4208] at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:411) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4208|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4209] at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:409) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4209|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4210] at org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:67) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4210|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4211] at org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:247) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4211|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4212] at org.apache.spark.sql.execution.streaming.ProcessingTimeExecutor.execute(TriggerExecutor.scala:67) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4212|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4213] at org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:237) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4213|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4214] at org.apache.spark.sql.execution.streaming.StreamExecution.$anonfun$runStream$1(StreamExecution.scala:306) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4214|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4215] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) ~[scala-library-2.12.17.jar:?]
> [4215|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4216] at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4216|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4217] at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:284) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4217|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4218] at org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:207) ~[spark-sql_2.12-3.4.3.jar:3.4.3]
> [4218|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4219]Caused by: java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes between first operation = \{actionType=commit, instantTime=20250114035518599, actionState=INFLIGHT'}, second operation = \{actionType=commit, instantTime=20250114035518542, actionState=COMPLETED'}
> [4219|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4220] ... 45 more
> [4220|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4221]2025-01-14 03:55:22.783:INFO:oejs.Server:main: jetty-9.4.53.v20231009; built: 2023-10-09T12:29:09.265Z; git: 27bde00a0b95a1d5bbee0eae7984f891d2d0f8c9; jvm 1.8.0_432-b06
> [4221|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4222]2025-01-14 03:55:22.785:INFO:oejs.Server:main: Started @1006285ms
> [4222|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4223]completeReplaceSize:1
> [4223|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4224]2025-01-14 03:55:36.149:INFO:oejs.Server:main: jetty-9.4.53.v20231009; built: 2023-10-09T12:29:09.265Z; git: 27bde00a0b95a1d5bbee0eae7984f891d2d0f8c9; jvm 1.8.0_432-b06
> [4224|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4225]2025-01-14 03:55:36.151:INFO:oejs.Server:main: Started @1019651ms
> [4225|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4226]completeReplaceSize:1
> [4226|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4227]Error: Tests run: 16, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 186.292 s <<< FAILURE! - in org.apache.hudi.functional.TestStructuredStreaming
> [4227|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4228]Error: testStructuredStreaming\{HoodieTableType}[1] Time elapsed: 6.777 s <<< ERROR!
> [4228|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4229]java.util.concurrent.ExecutionException: Boxed Error
> [4229|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4230] at scala.concurrent.impl.Promise$.resolver(Promise.scala:87)
> [4230|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4231] at scala.concurrent.impl.Promise$.scala$concurrent$impl$Promise$$resolveTry(Promise.scala:79)
> [4231|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4232] at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284)
> [4232|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4233] at scala.concurrent.Promise.complete(Promise.scala:53)
> [4233|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4234] at scala.concurrent.Promise.complete$(Promise.scala:52)
> [4234|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4235] at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:187)
> [4235|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4236] at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
> [4236|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4237] at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
> [4237|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4238] at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
> [4238|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4239] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> [4239|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4240] at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> [4240|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4241] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> [4241|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4242] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
> [4242|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4243]Caused by: java.lang.AssertionError: assertion failed
> [4243|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4244] at scala.Predef$.assert(Predef.scala:208)
> [4244|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4245] at org.apache.hudi.functional.TestStructuredStreaming.$anonfun$structuredStreamingTestRunner$1(TestStructuredStreaming.scala:151)
> [4245|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4246] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> [4246|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4247] at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
> [4247|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4248] at scala.util.Success.$anonfun$map$1(Try.scala:255)
> [4248|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4249] at scala.util.Success.map(Try.scala:213)
> [4249|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4250] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
> [4250|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4251] at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
> [4251|https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587#step:6:4252] ... 7 more
>
> Found in a PR that touches nothing on structured streaming and running the test locally has no issue
> https://github.com/apache/hudi/actions/runs/12760604136/job/35566295769?pr=12587
--
This message was sent by Atlassian Jira
(v8.20.10#820010)