You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Sam Whittle (Jira)" <ji...@apache.org> on 2020/08/25 13:45:00 UTC

[jira] [Created] (BEAM-10808) StreamingDataflowWorker streaming rpcs due not always observe stream failures, until timeout

Sam Whittle created BEAM-10808:
----------------------------------

             Summary: StreamingDataflowWorker streaming rpcs due not always observe stream failures, until timeout
                 Key: BEAM-10808
                 URL: https://issues.apache.org/jira/browse/BEAM-10808
             Project: Beam
          Issue Type: Bug
          Components: runner-dataflow
            Reporter: Sam Whittle
            Assignee: Sam Whittle


The StreamingDataflow worker communicates to Streaming Engine with grpc streaming rpcs. Occasionaly with service failures, the error is not observed by the streaming rpc until the deadline is exceeded.  As the deadline is multiple minutes, this adds unnecessary latency to pipelines that are otherwise keeping up with seconds delay.

Testing has shown that grpc channel health checks are not sufficient but that application level health checks ensure that streaming rpc health issues are detected earlier than the stream timeout.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)