You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Keith Lee (Jira)" <ji...@apache.org> on 2024/12/19 14:28:00 UTC

[jira] [Updated] (FLINK-36939) High CPU Utilization with Flink Kinesis EFO Consumer

     [ https://issues.apache.org/jira/browse/FLINK-36939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Keith Lee updated FLINK-36939:
------------------------------
    Attachment: Screenshot 1734584781285.png
                Screenshot 1734584639640.png

> High CPU Utilization with Flink Kinesis EFO Consumer
> ----------------------------------------------------
>
>                 Key: FLINK-36939
>                 URL: https://issues.apache.org/jira/browse/FLINK-36939
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Kinesis
>    Affects Versions: 1.20.0
>            Reporter: Keith Lee
>            Priority: Major
>         Attachments: Screenshot 1734584639640.png, Screenshot 1734584781285.png
>
>
> Observation: When EFO is enabled, the CPU usage spikes and stays elevated, regardless of record volume. If we switch back to the standard polling consumer (disabling EFO), CPU utilization returns to normal levels.
> Profiling Results: Local profiling and flamegraphs suggest the connector may be engaged in a busy-wait loop, continuously parking and un-parking threads even when no data is available. This behavior consumes CPU cycles unnecessarily.
> Performance Impact: While the job still processes records correctly when they arrive, the high baseline CPU consumption is concerning. It wastes resources and triggers unnecessary scaling, which doesn’t resolve the issue since new instances also experience the same CPU pattern.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)