You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Yanquan Lv (Jira)" <ji...@apache.org> on 2025/03/24 07:08:00 UTC

[jira] [Updated] (FLINK-37539) performance optimization in Paimon Sink to reduce end-to-end checkpoint time

     [ https://issues.apache.org/jira/browse/FLINK-37539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yanquan Lv updated FLINK-37539:
-------------------------------
    Attachment: image-2025-03-24-15-07-01-191.png

> performance optimization in Paimon Sink to reduce end-to-end checkpoint time
> ----------------------------------------------------------------------------
>
>                 Key: FLINK-37539
>                 URL: https://issues.apache.org/jira/browse/FLINK-37539
>             Project: Flink
>          Issue Type: Improvement
>          Components: Flink CDC
>    Affects Versions: cdc-3.3.0
>            Reporter: Yanquan Lv
>            Priority: Blocker
>         Attachments: image-2025-03-24-15-07-01-191.png
>
>
> At present, when using Paimon as a sink for jobs that require writing a large number of tables, the end-to-end checkpoint takes a long time because we need to perform commit operations at the checkpoint. 
> Perhaps we can use multithreading to have each table submit separately and then process them together.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)