[jira] [Comment Edited] (KAFKA-4246) Discretionary partition assignment on the consumer side not functional

["Vahid Hashemian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546229#comment-15546229 ] 

Vahid Hashemian edited comment on KAFKA-4246 at 10/4/16 6:29 PM:
-----------------------------------------------------------------

I'll try to run this in my environment to see if I can reproduce the issue. In the meantime, can you confirm if you have been able to reproduce the warning message in a fresh cluster?

Also, on your last point about the potential conflict between subscribing to topic and assigning partitions to the same consumer, there is at least [one unit test|https://github.com/apache/kafka/blob/a23859e5686bf93ed7e0d310f949757694d47a1b/clients/src/test/java/org/apache/kafka/clients/consumer/KafkaConsumerTest.java#L144] in the code that does both for the same consumer without an issue. The only thing necessary between the two actions is an {{unsubscribe()}} call.


was (Author: vahid):
I'll try to run this in my environment to see if I can reproduce the issue. In the meantime, can you confirm if you have been able to reproduce the warning message in a fresh cluster?

Also, on your last point about the potential conflict between subscribing to topic and assigning partitions to the same consumer, there is at least [one unit test|https://github.com/apache/kafka/blob/trunk/clients/src/test/java/org/apache/kafka/clients/consumer/KafkaConsumerTest.java#L144] in the code that does both for the same consumer without an issue. The only thing necessary between the two actions is an {{unsubscribe()}} call.

> Discretionary partition assignment on the consumer side not functional
> ----------------------------------------------------------------------
>
>                 Key: KAFKA-4246
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4246
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 0.10.0.1
>            Reporter: Alexandru Ionita
>
> Trying to manually assign partition/topics to a consumer will not work correctly. The consumer will be able to fetch records from the given partitions, but the first commit will fail with the following message:
> {code}
> 2016-10-03 13:44:50.673 DEBUG 11757 --- [pool-9-thread-1] o.a.k.c.c.internals.ConsumerCoordinator  : Offset commit for group XXXXXX failed: The coordinator is not aware of this member.
> 2016-10-03 13:44:50.673  WARN 11757 --- [pool-9-thread-1] o.a.k.c.c.internals.ConsumerCoordinator  : Auto offset commit failed for group XXXXXX: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.
> {code}.
> All this while the consumer will continue to poll records from the kafka cluster, but every commit will fail with the same message.
> I tried setting the {{session.timeout.ms}} to values like 50000, but I was getting the same outcome => no successfull commits.
> If I only switch from {{consumer.assign( subscribedPartitions )}} to {{consumer.subscribe( topics )}}, everything works as expected. No other client configurations should be changed to make it work.
> Am I missing something here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)