[jira] [Created] (SPARK-19564) KafkaOffsetReader's consumers should not be in the same group

["Liwei Lin (JIRA)" <ji...@apache.org>]

Liwei Lin created SPARK-19564:
---------------------------------

             Summary: KafkaOffsetReader's consumers should not be in the same group
                 Key: SPARK-19564
                 URL: https://issues.apache.org/jira/browse/SPARK-19564
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 2.1.1, 2.2.0
            Reporter: Liwei Lin
            Priority: Minor


In KafkaOffsetReader, when error occurs, we abort the existing consumer and create a new consumer. In our current implementation, the first consumer and the second consumer would be in the same group, which violates our intention of the two consumers not being in the same group.

The cause is that, in our current implementation, the first consumer is created before `groupId` and `nextId` are initialized in the constructor. Then even if `groupId` and `nextId` are increased during the creation of that first consumer, `groupId` and `nextId` would still be initialized to default values in the constructor.

We should make sure that `groupId` and `nextId` are initialized before the creation of the any consumer.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org