[jira] [Created] (KAFKA-13500) Consider adding a dedicated standby consumer

["Matthias J. Sax (Jira)" <ji...@apache.org>]

Matthias J. Sax created KAFKA-13500:
---------------------------------------

             Summary: Consider adding a dedicated standby consumer
                 Key: KAFKA-13500
                 URL: https://issues.apache.org/jira/browse/KAFKA-13500
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Matthias J. Sax


We currently use the restore consumer to recover state for active tasks and to maintain standby tasks during regular processing. This setup has a few disadvantages
 # During state recovery, we might want to apply different consumer configs compared to standby maintenance during regular processing.
 # It make monitoring confusing: because we never commit offsets for changelog topics, users can only monitor the client's "lag metric" to observer restore progress (without the need to register a restore listener). However, if they are interesting in a restore metric, during regular processing it would report the standby lag, which can be rather confusing.

Because the restore consumer does not use consumer group management, it seems to be low overhead to actually use a third consumer, because there won't be any heartbeat thread.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)