You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Jiayi Liao (Jira)" <ji...@apache.org> on 2020/11/21 09:34:00 UTC

[jira] [Created] (FLINK-20269) The flush happens too frequent in SavepoinV2Serializer

Jiayi Liao created FLINK-20269:
----------------------------------

             Summary: The flush happens too frequent in SavepoinV2Serializer
                 Key: FLINK-20269
                 URL: https://issues.apache.org/jira/browse/FLINK-20269
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Checkpointing
    Affects Versions: 1.9.0
            Reporter: Jiayi Liao


The reason I notice this is, I find that the metadata's persistence can be very slow, especially when the network is unstable, and almost every time I dump the stack of the process, the bottleneck happens on Hdfs client waiting for Datanodes' ack during metadata's persistance.

 

I wonder, is it really necessary to flush the stream after every {{StreamStateHandle}}'s serialization?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)