You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Harshit Pedamkar (Jira)" <ji...@apache.org> on 2024/12/13 12:56:00 UTC

[jira] [Created] (KAFKA-18233) Mirrormaker 2 All Worker tasks graceful shutdown fails after which the consumer group offsets are replicated .

Harshit Pedamkar created KAFKA-18233:
----------------------------------------

             Summary: Mirrormaker 2 All Worker tasks graceful shutdown fails after which the consumer group offsets are replicated .
                 Key: KAFKA-18233
                 URL: https://issues.apache.org/jira/browse/KAFKA-18233
             Project: Kafka
          Issue Type: Bug
          Components: mirrormaker
    Affects Versions: 3.5.1
            Reporter: Harshit Pedamkar
             Fix For: 3.5.1
         Attachments: connect-distributed.properties, connect.log, connect.log.2024-12-13-07, mm2-cpc-cust-repl-policy.json, mm2-hbc-cust-repl-policy.json, mm2-msc-cust-repl-policy.json

Hi,

It is noticed during load the  Checkpoint  Connectors graceful shutdown fails. After which the consumer group offsets are not replicates from source to target.

We have a distributed mirrormaker2 container running.  Logs are attached.

Logs snippets:

[2024-12-13 10:31:53,594] ERROR [mm2-cpc|worker] Connector 'mm2-cpc' failed to properly shut down, has become unresponsive, and may be consuming external resources. Correct the configuration for this connector or remove the connector. After fixing the connector, it may be necessary to restart this worker to release any consumed resources. (org.apache.kafka.connect.runtime.Worker:437)
[2024-12-13 10:31:58,818] ERROR [mm2-cpc|worker] Connector 'mm2-cpc' failed to properly shut down, has become unresponsive, and may be consuming external resources. Correct the configuration for this connector or remove the connector. After fixing the connector, it may be necessary to restart this worker to release any consumed resources. (org.apache.kafka.connect.runtime.Worker:437)
[2024-12-13 10:32:03,928] ERROR [mm2-cpc|worker] Connector 'mm2-cpc' failed to properly shut down, has become unresponsive, and may be consuming external resources. Correct the configuration for this connector or remove the connector. After fixing the connector, it may be necessary to restart this worker to release any consumed resources. (org.apache.kafka.connect.runtime.Worker:437)
[2024-12-13 10:32:30,444] ERROR [mm2-cpc|task-30] Graceful stop of task mm2-cpc-30 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,447] ERROR [mm2-cpc|task-27] Graceful stop of task mm2-cpc-27 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,449] ERROR [mm2-cpc|task-33] Graceful stop of task mm2-cpc-33 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,452] ERROR [mm2-cpc|task-36] Graceful stop of task mm2-cpc-36 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,455] ERROR [mm2-cpc|task-42] Graceful stop of task mm2-cpc-42 failed. (org.apache.kafka.connect.runtime.Worker:1029)

We have mirrormaker with the configuration. config files attached.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)