You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Hong Liang Teoh (Jira)" <ji...@apache.org> on 2025/06/26 16:38:00 UTC

[jira] [Updated] (FLINK-16839) Connectors Kinesis metrics can be disabled

     [ https://issues.apache.org/jira/browse/FLINK-16839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hong Liang Teoh updated FLINK-16839:
------------------------------------
    Fix Version/s: aws-connector-5.2.0
                       (was: aws-connector-5.1.0)

> Connectors Kinesis metrics can be disabled
> ------------------------------------------
>
>                 Key: FLINK-16839
>                 URL: https://issues.apache.org/jira/browse/FLINK-16839
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Kinesis, Runtime / Metrics
>    Affects Versions: 1.10.0
>            Reporter: Muchl
>            Priority: Not a Priority
>             Fix For: aws-connector-5.2.0
>
>
> Currently there are 9 metrics in the kinesis connector, each of which is recorded according to the kinesis shard dimension. If there are enough shards, taskmanager mtrics will be unavailable.
> In our production environment, a case of a job that reads dynamodb stream kinesis adapter, this dynamodb has more than 10,000 shards, multiplied by 9 metrics, there are more than 100,000 metrics for kinesis, and the entire metrics output reaches tens of MB , Cause prometheus cannot collect metrics.
> We should have a configuration item that can disable kinesis metrics



--
This message was sent by Atlassian Jira
(v8.20.10#820010)