[GitHub] [pulsar] willome opened a new issue #4672: [Kafka connect] Transformations are not working

[GitBox <gi...@apache.org>]

willome opened a new issue #4672: [Kafka connect] Transformations are not working
URL: https://github.com/apache/pulsar/issues/4672
 
 
   **Describe the bug**
   We can not use the Kafka Connect Transformations (https://docs.confluent.io/current/connect/transforms/index.html). This was tested with Debezium.
   
   **To Reproduce**
   From this : https://debezium.io/blog/2019/05/23/tutorial-using-debezium-connectors-with-apache-pulsar/
   
   1. Start Database
   ```
   docker run --rm --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=debezium -e MYSQL_USER=mysqluser -e MYSQL_PASSWORD=mysqlpw debezium/example-mysql:0.9
   ```
   2. Start Pulsar 
   ```bin/pulsar standalone```
   
   3. Start The debezium Mysql connector
   ```
   bin/pulsar-admin source localrun  --sourceConfigFile debezium-mysql-source-config.yaml
   ```
   with yaml 
   ```
   tenant: "test"
   namespace: "test-namespace"
   name: "debezium-kafka-source"
   topicName: "kafka-connect-topic"
   archive: "connectors/pulsar-io-kafka-connect-adaptor-2.3.0.nar"
   
   parallelism: 1
   
   configs:
     ## sourceTask
     task.class: "io.debezium.connector.mysql.MySqlConnectorTask"
     database.hostname: "localhost"
     database.port: "3306"
     database.user: "debezium"
     database.password: "dbz"
     database.server.id: "184054"
     database.server.name: "dbserver1"
     database.whitelist: "inventory"
     database.history: "org.apache.pulsar.io.debezium.PulsarDatabaseHistory"
     database.history.pulsar.topic: "history-topic"
     database.history.pulsar.service.url: "pulsar://127.0.0.1:6650"
     key.converter: "org.apache.kafka.connect.json.JsonConverter"
     value.converter: "org.apache.kafka.connect.json.JsonConverter"
     pulsar.service.url: "pulsar://127.0.0.1:6650"
     offset.storage.topic: "offset-topic"
     transforms: "unwrap"
     transforms.unwrap.type: "io.debezium.transforms.UnwrapFromEnvelope"
   ```
   4. Subscribe to topic
   ``` 
   bin/pulsar-client consume -s "sub-products" public/default/dbserver1.inventory.products -n 0
   ```
   
   **Expected behavior**
   messages from topic modified by the transformation (without the Enveloppe)
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services