[jira] [Created] (FLINK-38813) Enable Flink CDC connector to emit Debezium heartbeat events to downstream consumers

["Tejansh Rana (Jira)" <ji...@apache.org>]

Tejansh Rana created FLINK-38813:
------------------------------------

             Summary: Enable Flink CDC connector to emit Debezium heartbeat events to downstream consumers
                 Key: FLINK-38813
                 URL: https://issues.apache.org/jira/browse/FLINK-38813
             Project: Flink
          Issue Type: New Feature
          Components: Flink CDC
            Reporter: Tejansh Rana


Enable Flink CDC connectors to emit Debezium heartbeat events to downstream consumers, allowing applications to monitor database connection health and implement custom metrics for change data capture pipelines.

*Problem Statement:*

Flink CDC connectors currently skip heartbeat events generated by Debezium, making it difficult for downstream applications to:
 * Monitor database connection health and liveness
 * Track replication lag in real-time
 * Implement custom metrics for data streaming pipeline health
 * Detect stalled or failed binlog reading


The Flink CDC connectors process heartbeat events internally for offset tracking but do not emit them to downstream consumers.

 

*Proposed Solution:*

Expose heartbeat events as optional emitted records to enable downstream consumers to consume these events and implement their custom solutions.

This config would be false by default to allow backward compatibility. When set true by users, the Record Emitter will emit the heartbeat events in addition to the existing behaviour of updating the offsets.

I would be happy to create a PR for this feature should we accept this proposal.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)