[GitHub] [pulsar] rdhabalia commented on pull request #9035: [pulsar-io] Fix invalid topic name generation in kafka-source-connector

[GitBox <gi...@apache.org>]

rdhabalia commented on pull request #9035:
URL: https://github.com/apache/pulsar/pull/9035#issuecomment-757357337


   @sijie I didn't see your comment. 
   it's happening with below command where Invalid topic placeholder: `"topic.creation.classificationlog.include":"my-topic.public.classification",` 
   
   error:
   ```
   Getting the following ERRORs when trying to create producer using Pulsar debezium IO connector...
   21:22:41,780 ERROR [my-prop/my-cluster/debezium-postgres-source-0] [instance: 0] PulsarSink - Failed to create Producer while doing user publish
   org.apache.pulsar.client.api.PulsarClientException$InvalidTopicNameException: Invalid topic name: 'my-prop/my-cluster/ns/my-topic.public.classification'
   ```
   
   
   ```
   ./bin/pulsar-admin source localrun \
   --tenant my-prop \
   --namespace my-cluster/ns \
   --name debezium-postgres-source \
   --destination-topic-name persistent://my-prop/my-cluster/ns/debezium-postgres-topic-3 \
   --archive /root/apache-pulsar-2.6.2/pulsar-io-debezium-1.3.1-postgres-2.7.0-SNAPSHOT.nar \
   --classname org.apache.pulsar.io.debezium.postgres.DebeziumPostgresSource \
   --parallelism 1 \
   --processingGuarantees ATLEAST_ONCE \
   --broker-service-url pulsar+ssl://broker-uswest.com:6651 \
   --client-auth-params tlsCertFile:/cert.pem,tlsKeyFile:/key.pem \
   --client-auth-plugin org.apache.pulsar.client.impl.auth.AuthenticationTls \
   --tls-trust-cert-path /ca.pem \
   --use-tls \
   --source-config '
   
   {"connector.class":"io.debezium.connector.postgresql.PostgresConnector","tasks.max":1,"database.hostname":"ns-db.mail.bf1.yahoo.com","database.port":"1234","database.user":"test","database.password":"123","database.dbname":"ns","database.server.name":"ns","table.include.list":"public.classification","plugin.name":"pgoutput","message.key.columns":"public.classification:iprange","snapshot.mode":"always","slot.name":"cms_debezium","topic.creation.default.replication.factor":3,"topic.creation.default.partitions":5,"topic.creation.default.cleanup.policy":"compact","topic.creation.default.compression.type":"producer","topic.creation.groups":"classificationlog","topic.creation.classificationlog.include":"my-topic.public.classification","topic.creation.classificationlog.replication.factor":3,"topic.creation.classificationlog.partitions":5,"topic.creation.classificationlog.cleanup.policy":"compact","topic.creation.classificationlog.retention.ms":7776000000,"topic.creation.classificationlog
 .compression.type":"producer","pulsar.service.url":"pulsar+ssl://broker-uswest.com:6651","offset.storage.topic":"persistent://my-prop/my-cluster/ns/debezium-storage-topic-3","pulsar.auth.plugin":"org.apache.pulsar.client.impl.auth.AuthenticationTls","pulsar.auth.plugin.param":"tlsCertFile:/cert.pem,tlsKeyFile:/key.pem","pulsar.tls.insecure.connection":"true","pulsar.tls.trust.cert":"/ca.pem"}
   ```
   
   I think it also doesn't support cluster name in source config for which I will create a separate PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org