[jira] [Updated] (BEAM-6285) add parameters for offsetConsumer in KafkaIO.read()

["Xu Mingmin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xu Mingmin updated BEAM-6285:
-----------------------------
    Summary: add parameters for offsetConsumer in KafkaIO.read()  (was: option to disable offsetConsumer in KafkaIO)

> add parameters for offsetConsumer in KafkaIO.read()
> ---------------------------------------------------
>
>                 Key: BEAM-6285
>                 URL: https://issues.apache.org/jira/browse/BEAM-6285
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-kafka
>            Reporter: Xu Mingmin
>            Assignee: Xu Mingmin
>            Priority: Major
>
> Add an option in KafkaIO.read() to avoid using `offsetConsumer`. 
> *What's the problem:*
> When security is enabled, function `KafkaUnboundedReader.updateLatestOffsets()` always fails with lots of WARN log.
> *What's the cause:*
> `offsetConsumer` uses a mocked `ConsumerConfig.GROUP_ID_CONFIG`, and it doesn't work as expected when security is enabled. In our case, `ConsumerConfig.GROUP_ID_CONFIG` is the key to validate whether this consumer is approved.
> *What's the solution:*
> Add an option to disable `offsetConsumer` by developers;
> *What's the side effect:*
> With `offsetConsumer` disabled, backlog is unknown.
>  
> [~rangadi] any comments?
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)