You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Matthias J. Sax (Jira)" <ji...@apache.org> on 2026/08/11 18:50:00 UTC
[jira] [Updated] (KAFKA-20711) Streams task restore-remaining-records metric never reaches 0
[ https://issues.apache.org/jira/browse/KAFKA-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax updated KAFKA-20711:
------------------------------------
Fix Version/s: 4.2.2
> Streams task restore-remaining-records metric never reaches 0
> -------------------------------------------------------------
>
> Key: KAFKA-20711
> URL: https://issues.apache.org/jira/browse/KAFKA-20711
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.3.0
> Reporter: Nicholas Telford
> Assignee: Nicholas Telford
> Priority: Minor
> Fix For: 4.4.0, 4.2.2, 4.3.2
>
>
> The Kafka Streams Task-level metric {{restore-remaining-records}} is intended to track the total number of records that still need to be restored.
> This metric is inaccurate, never actually dropping to 0 for fully restored tasks, and always showing values substantially higher than reality, especially under EOS.
> The root-cause is that the metric is initialized with a total number of records to restored derived as {{logEndOffset - committedOffset}}, using a READ_UNCOMMITTED consumer.
> This offset range naturally includes uncommitted records and transaction markers, as well as offsets that have been compacted away, in addition to the actual records to restore.
> When decrementing the metric during restore, we decrement by the actual number of (committed) records that were restored. Since this excludes uncommitted records and transaction markers, we will never decrement the metric by the total it was initialized with.
> I have a fix that I will raise a PR for.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)