[jira] [Created] (KAFKA-6002) Kafka Connect Transform transforming JSON string into actual object

["Edvard Poliakov (JIRA)" <ji...@apache.org>]

Edvard Poliakov created KAFKA-6002:
--------------------------------------

             Summary: Kafka Connect Transform transforming JSON string into actual object
                 Key: KAFKA-6002
                 URL: https://issues.apache.org/jira/browse/KAFKA-6002
             Project: Kafka
          Issue Type: Improvement
          Components: KafkaConnect
            Reporter: Edvard Poliakov
            Priority: Minor


My colleague and I have been working on a new Transform, that takes a JSON string and transforms it into an actual object, like this:

{code} 
{
  "a" : "{\"b\": 23}"
}
{code}
into
{code}
{
  "a" : {
       "b" : 23
  }
}
{code}

There is no robust way of building a Schema from a JSON object itself, as it can be something like an empty array or a null, that doesn't provide any info on the schema of the object. So I see two options here.

1. For a transform to take in schema as a transform parameter. The problem I found with this is that it is not clear what JSON schema specification should be used for this? I assume it would be reasonable to use http://json-schema.org/, but it doesn't seem that Kafka Connect supports it currently, moreover reading through JsonConverter class in Kafka Connect, I am not able to understand what spec does the Json Schema have that is used in that class, for example {{asConnectSchema}} method on {{JsonConverte}}.

2. On each object received, keep updating the schema, but I can't see a standard and robust way of handling edge cases.

I am happy to create a pull request for this transform, if we can agree on something here. :)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)