You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2022/01/12 03:50:04 UTC
[jira] [Updated] (BEAM-3897) Flink runners fails on multioutput portable pipeline
[ https://issues.apache.org/jira/browse/BEAM-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenneth Knowles updated BEAM-3897:
----------------------------------
This Jira ticket has a pull request attached to it, but is still open. Did the pull request resolve the issue? If so, could you please mark it resolved? This will help the project have a clear view of its open issues.
> Flink runners fails on multioutput portable pipeline
> ----------------------------------------------------
>
> Key: BEAM-3897
> URL: https://issues.apache.org/jira/browse/BEAM-3897
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Henning Rohde
> Priority: P3
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When trying a Go pipeline with a 2-output DoFn on the hacking branch, I see:
> {noformat}
> [flink-runner-job-server] ERROR org.apache.beam.runners.flink.FlinkJobInvocation - Error during job invocation go-job-1521582585657843000_-2121541089_763230090.
> java.lang.RuntimeException: Pipeline execution failed
> at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:134)
> at org.apache.beam.runners.flink.FlinkJobInvocation.lambda$start$0(FlinkJobInvocation.java:61)
> at org.apache.beam.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
> at org.apache.beam.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
> at org.apache.beam.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply$mcV$sp(JobManager.scala:897)
> at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:840)
> at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:840)
> at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
> at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
> at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
> at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: expected one element but was: <0, 1>
> at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.lambda$mapPartition$0(FlinkExecutableStageFunction.java:176)
> at java.util.HashMap$Values.forEach(HashMap.java:980)
> at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.mapPartition(FlinkExecutableStageFunction.java:172)
> at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:490)
> at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:355)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: expected one element but was: <0, 1>
> at org.apache.beam.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:500)
> at org.apache.beam.util.concurrent.AbstractFuture.get(AbstractFuture.java:479)
> at org.apache.beam.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:76)
> at org.apache.beam.sdk.fn.data.SettableFutureInboundDataClient.awaitCompletion(SettableFutureInboundDataClient.java:41)
> at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion(BeamFnDataInboundObserver.java:89)
> at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.lambda$mapPartition$0(FlinkExecutableStageFunction.java:174)
> ... 7 more
> Caused by: java.lang.IllegalArgumentException: expected one element but was: <0, 1>
> at org.apache.beam.collect.Iterators.getOnlyElement(Iterators.java:322)
> at org.apache.beam.collect.Iterables.getOnlyElement(Iterables.java:294)
> at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction$1$1.accept(FlinkExecutableStageFunction.java:149)
> at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction$1$1.accept(FlinkExecutableStageFunction.java:135)
> at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.accept(BeamFnDataInboundObserver.java:80)
> at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.accept(BeamFnDataInboundObserver.java:32)
> at org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer$InboundObserver.onNext(BeamFnDataGrpcMultiplexer.java:135)
> at org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer$InboundObserver.onNext(BeamFnDataGrpcMultiplexer.java:123)
> at io.grpc.stub.ServerCalls$2$1.onMessage(ServerCalls.java:204)
> at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messageRead(ServerCallImpl.java:242)
> at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1.runInContext(ServerImpl.java:568)
> at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:52)
> at io.grpc.internal.SerializingExecutor$TaskRunner.run(SerializingExecutor.java:152)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.co
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)