[jira] [Created] (FLINK-15924) Detect and log blocking main thread operations

["Till Rohrmann (Jira)" <ji...@apache.org>]

Till Rohrmann created FLINK-15924:
-------------------------------------

             Summary: Detect and log blocking main thread operations
                 Key: FLINK-15924
                 URL: https://issues.apache.org/jira/browse/FLINK-15924
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Coordination
    Affects Versions: 1.10.0
            Reporter: Till Rohrmann
             Fix For: 1.11.0


When using the {{RpcEndpoint}} it is important that all operations which run on the main thread are never blocking. We have seen in the past that it is quite hard to always catch blocking operations in reviews and sometimes these changes caused instabilities in Flink. Once this happens it is not trivial to find the culprit which is responsible for the blocking operation.

One way to make debugging easier is to add a monitor which detects and logs if a {{RpcEndpoint}} operation takes longer than {{n}} seconds for example. Depending on the overhead of this monitor one could even think about enabling it only via a special configuration (e.g. debug mode).

A proper class to introduce this monitor could be the {{AkkRpcActor}} which is responsible for executing main thread operations. Whenever we schedule an operation, we could start a timeout which if triggered and the operation has not been completed will log a warning.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)