You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/10/22 10:07:00 UTC
[jira] [Updated] (FLINK-41054) Metrics of finished source splits are never removed from the REST API's metric store
[ https://issues.apache.org/jira/browse/FLINK-41054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated FLINK-41054:
-----------------------------------
Labels: pull-request-available (was: )
> Metrics of finished source splits are never removed from the REST API's metric store
> ------------------------------------------------------------------------------------
>
> Key: FLINK-41054
> URL: https://issues.apache.org/jira/browse/FLINK-41054
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics, Runtime / REST
> Affects Versions: 2.3.0, 2.2.1, 2.1.3, 2.3.1
> Reporter: Diljeet Singh
> Priority: Major
> Labels: pull-request-available
>
> InternalSourceSplitMetricGroup registers per-split metrics under split.<splitId>.watermark.*, for example currentWatermark, pausedTimePerSecond, idleTimePerSecond and activeTimePerSecond. When a split finishes, SourceOperator#splitFinished closes the group. That unregisters the metrics on the TaskManager, so they no longer appear in its MetricQueryService dump.
> The JobManager's MetricStore only merges each fetched dump into its existing maps. It removes whole jobs, TaskManagers, subtasks and old attempts, but never an individual metric that stops being reported. The finished splits' metrics therefore stay in the store with their last values. As a result, endpoints such as /jobs/:jobid/vertices/:vertexid/subtasks/metrics (and the subtask and attempt metric endpoints) list a growing number of metrics for each split ever processed. For sources with many short-lived splits, such as file or Kafka sources with partition discovery, this grows without bound. It increases JobManager memory use, makes REST responses slower, and clutters the metric picker in the web UI with stale entries.
> Steps to reproduce:
> 1. Run a job with a FLIP-27 source that processes many splits over time, such as a FileSource in continuous monitoring mode.
> 2. Call GET /jobs/:jobid/vertices/:vertexid/subtasks/metrics repeatedly.
> 3. The returned list keeps growing with split.<id>.watermark.* entries for splits that have already finished.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)