You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Alexey Romanenko (JIRA)" <ji...@apache.org> on 2019/01/30 09:31:00 UTC
[jira] [Resolved] (BEAM-6285) add parameters for offsetConsumer in
KafkaIO.read()
[ https://issues.apache.org/jira/browse/BEAM-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexey Romanenko resolved BEAM-6285.
------------------------------------
Resolution: Fixed
Fix Version/s: 2.11.0
> add parameters for offsetConsumer in KafkaIO.read()
> ---------------------------------------------------
>
> Key: BEAM-6285
> URL: https://issues.apache.org/jira/browse/BEAM-6285
> Project: Beam
> Issue Type: Improvement
> Components: io-java-kafka
> Reporter: Xu Mingmin
> Assignee: Xu Mingmin
> Priority: Major
> Fix For: 2.11.0
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Add an option in KafkaIO.read() to avoid using `offsetConsumer`.
> *What's the problem:*
> When security is enabled, function `KafkaUnboundedReader.updateLatestOffsets()` always fails with lots of WARN log.
> *What's the cause:*
> `offsetConsumer` uses a mocked `ConsumerConfig.GROUP_ID_CONFIG`, and it doesn't work as expected when security is enabled. In our case, `ConsumerConfig.GROUP_ID_CONFIG` is the key to validate whether this consumer is approved.
> *What's the solution:*
> Add an option to disable `offsetConsumer` by developers;
> *What's the side effect:*
> With `offsetConsumer` disabled, backlog is unknown.
>
> [~rangadi] any comments?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)