You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "sunjincheng (Jira)" <ji...@apache.org> on 2019/11/18 01:59:00 UTC
[jira] [Commented] (BEAM-8733) The "KeyError: u'-47'" error from
line 305 of sdk_worker.py
[ https://issues.apache.org/jira/browse/BEAM-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976225#comment-16976225 ]
sunjincheng commented on BEAM-8733:
-----------------------------------
Hi, Thanks for the log info, [~chamikara].
From the exception log(the line number of RegisterAndProcessBundleOperation), it seems that the [commit|https://github.com/apache/beam/commit/686833381ecc92f0fbe04e576a582a7640ca7bbd] is not included in the DataFlow runner. Could you help to check this as this commit ensures that registration is executed successfully before executing process bundle request?
> The "KeyError: u'-47'" error from line 305 of sdk_worker.py
> -----------------------------------------------------------
>
> Key: BEAM-8733
> URL: https://issues.apache.org/jira/browse/BEAM-8733
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: sunjincheng
> Priority: Major
> Fix For: 2.18.0
>
>
> The issue reported by [~chamikara], error message as follows:
> apache_beam/runners/worker/sdk_worker.py", line 305, in get
> self.fns[bundle_descriptor_id],
> KeyError: u'-47'
> {code}
> at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:57)
> at org.apache.beam.runners.dataflow.worker.fn.control.RegisterAndProcessBundleOperation.finish(RegisterAndProcessBundleOperation.java:330)
> at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:85)
> at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:125)
> at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:411)
> at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:380)
> at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:305)
> at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
> at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
> Suppressed: java.lang.IllegalStateException: Already closed.
> at org.apache.beam.sdk.fn.data.BeamFnDataBufferingOutboundObserver.close(BeamFnDataBufferingOutboundObserver.java:93)
> at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.abort(RemoteGrpcPortWriteOperation.java:220)
> at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:91)
> {code}
> More discussion info can be found here: https://github.com/apache/beam/pull/10004
--
This message was sent by Atlassian Jira
(v8.3.4#803005)