You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "eaugene (via GitHub)" <gi...@apache.org> on 2023/08/24 18:36:59 UTC
[GitHub] [kafka] eaugene opened a new pull request, #14287: [Minor] Check the existence of AppInfo for the given ID before creating a new mbean of the same name
eaugene opened a new pull request, #14287:
URL: https://github.com/apache/kafka/pull/14287
When using kafka consumer in apache pinot , we did see couple of WARN as we are trying to create kafka consumer class with the same name . We currently have to use a added suffix to create a new mBean as each new kafka consumer in same process creates a mBean . Adding support here to skip creation of mBean if its already existing
```
javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=transcript2_REALTIME-transcript-topic-0
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:436) ~[?:?]
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) ~[?:?]
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) ~[?:?]
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) ~[?:?]
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) ~[?:?]
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) ~[?:?]
at org.apache.pinot.shaded.org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:64) [pinot-protobuf-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-02fe2f1f864627b4a0e80c1291422b75cd173879]
at org.apache.pinot.shaded.org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:814) [pinot-protobuf-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-02fe2f1f864627b4a0e80c1291422b75cd173879]
at org.apache.pinot.shaded.org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:665) [pinot-protobuf-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-02fe2f1f864627b4a0e80c1291422b75cd173879]
at org.apache.pinot.shaded.org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646) [pinot-protobuf-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-02fe2f1f864627b4a0e80c1291422b75cd173879]
at org.apache.pinot.shaded.org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:626) [pinot-protobuf-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-02fe2f1f864627b4a0e80c1291422b75cd173879]
at org.apache.pinot.plugin.stream.kafka20.KafkaPartitionLevelConnectionHandler.createConsumer(KafkaPartitionLevelConnectionHandler.java:84) [pinot-kafka-2.0-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-02fe2f1f864627b4a0e80c1291422b75cd173879]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] eaugene commented on a diff in pull request #14287: [Minor] Check the existence of AppInfo for the given ID before creating a new mbean of the same name
Posted by "eaugene (via GitHub)" <gi...@apache.org>.
eaugene commented on code in PR #14287:
URL: https://github.com/apache/kafka/pull/14287#discussion_r1319598294
##########
clients/src/main/java/org/apache/kafka/common/utils/AppInfoParser.java:
##########
@@ -60,8 +60,13 @@ public static String getCommitId() {
public static synchronized void registerAppInfo(String prefix, String id, Metrics metrics, long nowMs) {
try {
ObjectName name = new ObjectName(prefix + ":type=app-info,id=" + Sanitizer.jmxSanitize(id));
+ MBeanServer server = ManagementFactory.getPlatformMBeanServer();
+ if (server.isRegistered(name)) {
+ log.info("App info {} for {} already exists, so skipping a new mbean creation", prefix, id);
Review Comment:
Yes , you are correct . I have modified
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] eaugene commented on pull request #14287: [Minor] Check the existence of AppInfo for the given ID before creating a new mbean of the same name
Posted by "eaugene (via GitHub)" <gi...@apache.org>.
eaugene commented on PR #14287:
URL: https://github.com/apache/kafka/pull/14287#issuecomment-1696882205
@satishd Can you please help review this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] eaugene commented on pull request #14287: [Minor] Check the existence of AppInfo for the given ID before creating a new mbean of the same name
Posted by "eaugene (via GitHub)" <gi...@apache.org>.
eaugene commented on PR #14287:
URL: https://github.com/apache/kafka/pull/14287#issuecomment-1711400368
> @eaugene , thanks for the fix. Left a comment, also, please add a test for this change. Thanks.
@showuon I have added a test case for this . The `InstanceAlreadyExistsException` is currently caught in by `JMException` in AppInfoParser .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon merged pull request #14287: MINOR: Check the existence of AppInfo for the given ID before creating a new mbean of the same name
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon merged PR #14287:
URL: https://github.com/apache/kafka/pull/14287
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #14287: [Minor] Check the existence of AppInfo for the given ID before creating a new mbean of the same name
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on PR #14287:
URL: https://github.com/apache/kafka/pull/14287#issuecomment-1718864019
Failed tests are unrelated:
```
Build / JDK 20 and Scala 2.13 / org.apache.kafka.connect.integration.ConnectorRestartApiIntegrationTest.testMultiWorkerRestartOnlyConnector
Build / JDK 20 and Scala 2.13 / kafka.server.DescribeClusterRequestTest.testDescribeClusterRequestExcludingClusterAuthorizedOperations(String).quorum=kraft
Build / JDK 20 and Scala 2.13 / org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()
Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testReplicateSourceDefault()
Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.integration.ConnectorRestartApiIntegrationTest.testMultiWorkerRestartOnlyConnector
Build / JDK 17 and Scala 2.13 / kafka.server.DescribeClusterRequestTest.testDescribeClusterRequestIncludingClusterAuthorizedOperations(String).quorum=kraft
Build / JDK 17 and Scala 2.13 / org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()
Build / JDK 11 and Scala 2.13 / kafka.api.SaslScramSslEndToEndAuthorizationTest.testAuthentications(String).quorum=zk
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a diff in pull request #14287: [Minor] Check the existence of AppInfo for the given ID before creating a new mbean of the same name
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #14287:
URL: https://github.com/apache/kafka/pull/14287#discussion_r1319285955
##########
clients/src/main/java/org/apache/kafka/common/utils/AppInfoParser.java:
##########
@@ -60,8 +60,13 @@ public static String getCommitId() {
public static synchronized void registerAppInfo(String prefix, String id, Metrics metrics, long nowMs) {
try {
ObjectName name = new ObjectName(prefix + ":type=app-info,id=" + Sanitizer.jmxSanitize(id));
+ MBeanServer server = ManagementFactory.getPlatformMBeanServer();
+ if (server.isRegistered(name)) {
+ log.info("App info {} for {} already exists, so skipping a new mbean creation", prefix, id);
Review Comment:
nit: additional space after _skipping_.
Question: What is this log output under consumer's case? I think it's:
`App info kafka.consumer for transcript2_REALTIME-transcript-topic-0 already exists, so skipping a new mbean creation`
Could we change to:
`log.info("The mbean of App info: [{}], id: [{}] already exists, so skipping a new mbean creation.", prefix, id);`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a diff in pull request #14287: [Minor] Check the existence of AppInfo for the given ID before creating a new mbean of the same name
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #14287:
URL: https://github.com/apache/kafka/pull/14287#discussion_r1319632459
##########
clients/src/main/java/org/apache/kafka/common/utils/AppInfoParser.java:
##########
@@ -60,8 +60,13 @@ public static String getCommitId() {
public static synchronized void registerAppInfo(String prefix, String id, Metrics metrics, long nowMs) {
try {
ObjectName name = new ObjectName(prefix + ":type=app-info,id=" + Sanitizer.jmxSanitize(id));
+ MBeanServer server = ManagementFactory.getPlatformMBeanServer();
+ if (server.isRegistered(name)) {
+ log.info("App info {} for {} already exists, so skipping a new mbean creation", prefix, id);
Review Comment:
Thanks. Please add tests for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org