[GitHub] [beam] thdesc commented on issue #21598: Beam worker closing gRPC connection with many workers and large shuffle sizes

["thdesc (via GitHub)" <gi...@apache.org>]

thdesc commented on issue #21598:
URL: https://github.com/apache/beam/issues/21598#issuecomment-1525306893

   According to this [documentation](https://github.com/grpc/grpc/blob/master/doc/keepalive.md), the server may send GOAWAY with ENHANCE_YOUR_CALM to the client if "the client's `GRPC_ARG_KEEPALIVE_TIME_MS` setting is lower than the server's `GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS`." Therefore, it seems that we should consider decreasing the value of `GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS` to 19 seconds, instead of modifying the `KEEP_ALIVE_TIME_SEC` on the server side. What do you think @HuangXingBo ? Alternatively, we could increase the value of `grpc.keepalive_time_ms` in the `channel_factory.py` to a value higher than `GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS`, which is set to 300 000 ms by default. In my case, I chose to set it to 300 001 ms, and I have not encountered the error again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org