[jira] [Commented] (KAFKA-5538) User-specified Producer/Consumer config doesn't effect with KafkaBackingStore(Config/Status/Offset)

["Ewen Cheslack-Postava (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175839#comment-16175839 ] 

Ewen Cheslack-Postava commented on KAFKA-5538:
----------------------------------------------

[~Aegeaner] I like the idea, but let me explain my original thinking around the configs. The three "sets" of configs (unprefixed, producer., and consumer.) are supposed to map to "framework stuff", "source stuff", and "sink stuff". In that context, the producer/consumer used by each of the internal logs is more of a framework/worker-level concern, whereas the prefixed configs are used for individual tasks.

The reason for making this distinction is that you might want the traffic for the framework vs source tasks vs consumer tasks to use different principals for applying different ACLs, different client IDs to enforce quotas, etc. For example, you might want the framework to use a principal that includes access to Connect internal topics whereas you might want the source/sink tasks to have more restricted access (especially if you are providing Connect as a service where users can instantiate their own connectors, but you manage the framework/service).

Does that make sense? Given that, is there a gap for what you're trying to accomplish?

> User-specified Producer/Consumer config doesn't effect with KafkaBackingStore(Config/Status/Offset)
> ---------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5538
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5538
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 0.11.0.0
>            Reporter: Aegeaner
>            Assignee: Aegeaner
>
> For configuration of Kafka source and Kafka sink tasks, the same parameters can be used but need to be prefixed with consumer. and producer. The worker will take off the prefix and get user specified configurations, but the KafkaBackingStores will not. All the three KafkaBackingStores just took originals from the Kafka config without taking off the prefix, so the producer/consumer will ignore these configurations. (e.g. Kerberos configs)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)