[jira] [Created] (FLINK-37436) flink-connector-pulsar used a incorrect API when dynamically creating topics by DynamicTopicRouter

["Yubiao Feng (Jira)" <ji...@apache.org>]

Yubiao Feng created FLINK-37436:
-----------------------------------

             Summary: flink-connector-pulsar used a incorrect API when dynamically creating topics by DynamicTopicRouter
                 Key: FLINK-37436
                 URL: https://issues.apache.org/jira/browse/FLINK-37436
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Pulsar
            Reporter: Yubiao Feng


Background 1: dynamically create a Pulsar Topic by Flink connector-pulsar

Flink connector-pulsar provided a way to dynamically create a Pulsar Topic when DynamicTopicRouter returns a non-existing one. see also: [flink-connector-pulsar/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java at main · apache/flink-connector-pulsar|https://github.com/apache/flink-connector-pulsar/blob/main/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java#L221]. 
 * `pulsarClient.getPartitionsForTopic(topic)` will create a topic automatically if it does not exist.

---

 

Background 2:  how dynamically created topics in Pulsar Server

- There is a config named `allowAutoTopicCreationType`, which can be set to `
partitioned` or `non-partitioned`
- If it was set `partitioned`, Pulsar will create a partitioned topic with `{defaultNumPartitions}` partitions. For example, Pulsar will create topics named `\{tenant}/\{namespace}/\{topic name}-partition-0` and `\{tenant}/\{namespace}/\{topic name}-partition-1`, and create a relationship between them, which indicates they are in a same partitioned topic.
- If it was set `non-partitioned`, Pulsar will create a non-partitioned topic. Pulsar will create topics named `\{tenant}/\{namespace}/\{topic name}`, which does not include a suffix `partition-\{num}`
 
---
 
Issue:
- if `pulsarClient.getPartitionsForTopic(topic)` get a param `\{tenant}/\{namespace}/\{topic name}-partition-0`, which includes the suffix `partition-0`, Pulsar will create a non-partitioned topic named `\{tenant}/\{namespace}/\{topic name}-partition-0`
- After you call `pulsarClient.getPartitionsForTopic(topic)` with a param `\{tenant}/\{namespace}/\{topic name}-partition-1`, you will get two partitions named `\{tenant}/\{namespace}/\{topic name}-partition-0` and `\{tenant}/\{namespace}/\{topic name}-partition-1`, but there is no relationship record between them.
 
 
 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)