You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Raj (Jira)" <ji...@apache.org> on 2019/11/12 09:02:00 UTC
[jira] [Updated] (KAFKA-9172) Kafka Connect JMX : source & sink
task metrics are missing in some cases after rebalancing of the tasks
[ https://issues.apache.org/jira/browse/KAFKA-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raj updated KAFKA-9172:
-----------------------
Description:
Kafka Connect exposes various metrics via JMX. We observed some times that few of the sink task metrics mbeans are getting deleted just after workers rebalances all the tasks.
Also, I don't see any logs getting registered related to sink-task-metrics mbeans at the same time . But I see similar WARN log at same time :
{code:java}
2019-11-11 20:58:09 WARN AppInfoParser:66 - Error registering AppInfo mbean
javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=ResiliencyRestartJob90
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:62)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:784)
at org.apache.kafka.connect.runtime.WorkerSinkTask.createConsumer(WorkerSinkTask.java:481)
at org.apache.kafka.connect.runtime.WorkerSinkTask.initialize(WorkerSinkTask.java:140)
at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:452)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:865)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:110)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:880)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:876)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
Please ask me if you need any additional information.
was:
Kafka Connect exposes various metrics via JMX. We observed some times that few of the sink task metrics mbeans are getting deleted just after workers rebalances all the tasks.
Also, I don't see any logs getting registered related to sink-task-metrics mbeans at the same time . But I see similar WARN log at same time :
{code:java}
// 2019-11-11 20:58:09 WARN AppInfoParser:66 - Error registering AppInfo mbean
javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=ResiliencyRestartJob90
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:62)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:784)
at org.apache.kafka.connect.runtime.WorkerSinkTask.createConsumer(WorkerSinkTask.java:481)
at org.apache.kafka.connect.runtime.WorkerSinkTask.initialize(WorkerSinkTask.java:140)
at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:452)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:865)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:110)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:880)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:876)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
Please ask me if you need any additional information.
> Kafka Connect JMX : source & sink task metrics are missing in some cases after rebalancing of the tasks
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-9172
> URL: https://issues.apache.org/jira/browse/KAFKA-9172
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.1.1
> Reporter: Raj
> Priority: Major
>
> Kafka Connect exposes various metrics via JMX. We observed some times that few of the sink task metrics mbeans are getting deleted just after workers rebalances all the tasks.
> Also, I don't see any logs getting registered related to sink-task-metrics mbeans at the same time . But I see similar WARN log at same time :
>
> {code:java}
> 2019-11-11 20:58:09 WARN AppInfoParser:66 - Error registering AppInfo mbean
> javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=ResiliencyRestartJob90
> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
> at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:62)
> at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:784)
> at org.apache.kafka.connect.runtime.WorkerSinkTask.createConsumer(WorkerSinkTask.java:481)
> at org.apache.kafka.connect.runtime.WorkerSinkTask.initialize(WorkerSinkTask.java:140)
> at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:452)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:865)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:110)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:880)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:876)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
> Please ask me if you need any additional information.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)