You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "fanrui (Jira)" <ji...@apache.org> on 2022/03/21 04:59:00 UTC

[jira] [Created] (FLINK-26759) Legacy source support waiting for recordWriter to be available

fanrui created FLINK-26759:
------------------------------

             Summary: Legacy source support waiting for recordWriter to be available
                 Key: FLINK-26759
                 URL: https://issues.apache.org/jira/browse/FLINK-26759
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Common, Runtime / Checkpointing
    Affects Versions: 1.14.0, 1.13.0, 1.15.0
            Reporter: fanrui
             Fix For: 1.16.0


In order for Unaligned Checkpoint not to be blocked, StreamTask#processInput will check recordWriter.isAvailable(). If not available, the data will not be processed until recordWriter is available.

The new Source api is compatible with the above logic, but Legacy Source is not compatible with the above logic. When using Unaligned Checkpoint, if the backpressure of Legacy Source is high, the Checkpoint duration of Legacy Source will be very long.

 

Since legacy sources are often used in production, can we add logic to wait for recordWriter to be available for legacy source?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)