You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Martijn Visser (Jira)" <ji...@apache.org> on 2022/03/04 11:05:00 UTC
[jira] [Commented] (FLINK-26379) First message produced via Flink Kafka Connector is slow
[ https://issues.apache.org/jira/browse/FLINK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501267#comment-17501267 ]
Martijn Visser commented on FLINK-26379:
----------------------------------------
[~rkrishnamurthy-rf] Flink 1.14 still supports Scala 2.11 and newer Kafka versions had dropped support for Scala 2.11, which blocked the upgrade. In Flink 1.15 it has been upgraded to 2.8 since it also dropped Scala 2.11 support.
> First message produced via Flink Kafka Connector is slow
> --------------------------------------------------------
>
> Key: FLINK-26379
> URL: https://issues.apache.org/jira/browse/FLINK-26379
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
> Affects Versions: 1.14.3
> Reporter: Ragu Krishnamurthy
> Priority: Major
>
> Topic partition map is initialized when the first message is processed rather then constructor
>
> *Code at constructor*
> [https://github.com/apache/flink/blob/5c6a220cc5e63b3c308e394b9199fb7740c08539/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer.java#L740]
> {code:java}
> this.topicPartitionsMap = new HashMap<>(); {code}
>
> *Code at invoke method*
> [https://github.com/apache/flink/blob/5c6a220cc5e63b3c308e394b9199fb7740c08539/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer.java#L879]
> {code:java}
> int[] partitions = topicPartitionsMap.get(targetTopic);
> if (null == partitions) {
> partitions = getPartitionsByTopic(targetTopic, transaction.producer); topicPartitionsMap.put(targetTopic, partitions);
> } {code}
> Can this behavior be changed?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)