You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ryan Tomczik (Jira)" <ji...@apache.org> on 2020/03/28 13:30:00 UTC

[jira] [Created] (KAFKA-9782) Kafka Connect InsertField transform - Add the ability to insert event's Key into Value

Ryan Tomczik created KAFKA-9782:
-----------------------------------

             Summary: Kafka Connect InsertField transform - Add the ability to insert event's Key into Value
                 Key: KAFKA-9782
                 URL: https://issues.apache.org/jira/browse/KAFKA-9782
             Project: Kafka
          Issue Type: Improvement
          Components: KafkaConnect
            Reporter: Ryan Tomczik


I'm using Debezium to pull change data capture events from a Mongo DB and write them to S3 with the Confluent S3 Sink. The problem is Debezium stores the document's key in each event's key and the S3 connector discards this key. I need the ability to insert the key as a new field in the event value. It seems that this would fit in perfectly into the InsertField transform or create a new transform KeyToValue.

Here is an example of someone else running into this same limitation and creating a custom transform.
[https://gist.github.com/shashidesai/aaf72489165c6a0fd73a3b51e5a8892a]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)