[jira] [Commented] (KAFKA-7723) Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api

["Randall Hauch (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793635#comment-16793635 ] 

Randall Hauch commented on KAFKA-7723:
--------------------------------------

[~laomei], first of all, thanks for logging this issue, creating [KIP-407|https://cwiki.apache.org/confluence/display/KAFKA/KIP-407%3A+Kafka+Connect+support+override+worker+kafka+api+configuration+with+connector+configuration+that+post+by+rest+api], and creating a pull request.

However, I think this is nearly identical to (or rather a subset of) KAFKA-6890 / [KIP-296|https://cwiki.apache.org/confluence/display/KAFKA/KIP-296%3A+Connector+level+configurability+for+client+configs], which IMO has the correct scope and where a discussion is taking place about the requirements and user experience. This proposal seems to differ from KAFKA-6890 / KIP-296 is that the approach proposed here only addresses connector configurations specified via the REST API, and not via configuration files passed to the standalone Connect worker. This would be a significant departure from the current behavior, where the REST API and file configurations are completely compatible. 

Since KAFKA-6890 / KIP-296 are older, can we resolve this issue as DUPLICATE, close the PR without merging, and withdraw KIP-407?

> Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-7723
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7723
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: laomei
>            Priority: Minor
>              Labels: needs-kip
>
> I'm using kafka sink connect; "auto.offset.reset" is set in connect-distributed*.properties; 
> It works for all connector which in one worker; So the consumer will poll records from latest or earliest; I can not control the auto.offset.reset in connector configs post with rest api;
> So I think is necessary to override worker kafka api configs with connector configs;  
> Like this
> {code:java}
>   {
>     "name": "test",
>     "config": {
>         "consumer.auto.offset.reset": "latest",
>         "consumer.xxx"
>         "connector.class": "com.laomei.sis.solr.SolrConnector",
>         "tasks.max": "1",
>         "poll.interval.ms": "100",
>         "connect.timeout.ms": "60000",
>         "topics": "test"
>     }
>   }
> {code}
> We can override kafka consumer auto offset reset in sink connector;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)