[jira] [Commented] (KAFKA-7273) Converters should have access to headers.

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782952#comment-16782952 ] 

ASF GitHub Bot commented on KAFKA-7273:
---------------------------------------

sap1ens commented on pull request #6362: KAFKA-7273: [WIP] extend Connect Converter to support headers
URL: https://github.com/apache/kafka/pull/6362
 
 
   Extending `Converter` interface to support headers in backwards-compatible way. Very similar to [Serializer](https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/serialization/Serializer.java#L61-L63) and [Deserializer](https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/serialization/Deserializer.java) interfaces.
   
   A few questions I have to the contributors:
   - Do I need to update internal components like KafkaStatusBackingStore that use Converters? It doesn't look like they use headers in the moment, so I don't think it's necessary.
   - How about other Converter implementations like JsonConverter? Should they all be switched to the new interface method even if they don't use the headers?
   - Is KIP required in this case? Converter _is_ a public interface, but the change is trivial and backwards-compatible. 
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Converters should have access to headers.
> -----------------------------------------
>
>                 Key: KAFKA-7273
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7273
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: Jeremy Custenborder
>            Assignee: Jeremy Custenborder
>            Priority: Major
>
> I found myself wanting to build a converter that stored additional type information within headers. The converter interface does not allow a developer to access to the headers in a Converter. I'm not suggesting that we change the method for serializing them, rather that *org.apache.kafka.connect.header.Headers* be passed in for *fromConnectData* and *toConnectData*. For example something like this.
> {code:java}
> import org.apache.kafka.connect.data.Schema;
> import org.apache.kafka.connect.data.SchemaAndValue;
> import org.apache.kafka.connect.header.Headers;
> import org.apache.kafka.connect.storage.Converter;
> public interface Converter {
>   default byte[] fromConnectData(String topic, Headers headers, Schema schema, Object object) {
>     return fromConnectData(topic, schema, object);
>   }
>   default SchemaAndValue toConnectData(String topic, Headers headers, byte[] payload) {
>     return toConnectData(topic, payload);
>   }
>   void configure(Map<String, ?> var1, boolean var2);
>   byte[] fromConnectData(String var1, Schema var2, Object var3);
>   SchemaAndValue toConnectData(String var1, byte[] var2);
> }
> {code}
> This would be a similar approach to what was already done with ExtendedDeserializer and ExtendedSerializer in the Kafka client.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)