You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Leonid Ilyevsky (JIRA)" <ji...@apache.org> on 2019/05/17 20:54:00 UTC

[jira] [Created] (FLINK-12548) FlinkKafkaConsumer issues configuring underlying KafkaConsumer

Leonid Ilyevsky created FLINK-12548:
---------------------------------------

             Summary: FlinkKafkaConsumer issues configuring underlying KafkaConsumer
                 Key: FLINK-12548
                 URL: https://issues.apache.org/jira/browse/FLINK-12548
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kafka
    Affects Versions: 1.8.0
            Reporter: Leonid Ilyevsky
         Attachments: KafkaError.txt

I observe two issues that are possibly related, or at least belong to the same part of the code.

I am running multiple instances of pipelines (by setting parallelism) under Flink cluster.

First problem I see is the "Error registering AppInfo mbean", see attached stack trace [^KafkaError.txt] . From the trace I understand that it tries to use consumer group id as the bean name; obviously this fails as I have multiple instances using the same group id under the same task manager.

Second problem is, the client id is not set at all (I do provide the "client.id" property when invoking FlinkKafkaConsumer constructor). This creates problems tracking consumers on Kafka side. On the other hand, I obviously do not want to set the same client id on all instances - preferably Flink should add maybe some UUID suffix to it or something to make it unique.

The same unique name could be used for that mbean, this way solving the first problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)