[jira] [Created] (BEAM-9630) Python SDK streaming from Pubsub getting error `grpc.StatusRuntimeException: CANCELLED: call already cancelled`

["Kan Dong (Jira)" <ji...@apache.org>]

Kan Dong created BEAM-9630:
------------------------------

             Summary: Python SDK streaming from Pubsub getting error `grpc.StatusRuntimeException: CANCELLED: call already cancelled`
                 Key: BEAM-9630
                 URL: https://issues.apache.org/jira/browse/BEAM-9630
             Project: Beam
          Issue Type: Bug
          Components: runner-dataflow
    Affects Versions: 2.19.0
         Environment: python==3.7.5
apache-beam[gcp]==2.19.0
google-cloud-pubsub==1.4.2
            Reporter: Kan Dong


I have a dataflow streaming job using Apache Beam Python 3.7 SDK 2.19.0.  The job consumes pubsub messages, treat data and publish to pubsub as output.  Periodically, I would get the below error messages and the worker would stop consuming messages.

```Error message from worker: org.apache.beam.vendor.grpc.v1p21p0.io.grpc.StatusRuntimeException: CANCELLED: call already cancelled
 org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Status.asRuntimeException(Status.java:524)
 org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onNext(ServerCalls.java:339)
 org.apache.beam.sdk.fn.stream.DirectStreamObserver.onNext(DirectStreamObserver.java:98)
 org.apache.beam.sdk.fn.stream.SynchronizedStreamObserver.onNext(SynchronizedStreamObserver.java:46)
 org.apache.beam.runners.fnexecution.control.FnApiControlClient.handle(FnApiControlClient.java:84)
 org.apache.beam.runners.dataflow.worker.fn.control.RegisterAndProcessBundleOperation.start(RegisterAndProcessBundleOperation.java:299)
 org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:77)
 org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:125)
 org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1350)
 org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$1100(StreamingDataflowWorker.java:152)
 org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$7.run(StreamingDataflowWorker.java:1073)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 java.lang.Thread.run(Thread.java:748)```



--
This message was sent by Atlassian Jira
(v8.3.4#803005)