[GitHub] [spark] xuanyuanking commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

[GitBox <gi...@apache.org>]

xuanyuanking commented on pull request #29729:
URL: https://github.com/apache/spark/pull/29729#issuecomment-716444949


   Thanks @gaborgsomogyi for the solid work! Leave some investigations on my side:
   
   - For `group.id` based authorization: The exception, `org.apache.kafka.common.errors.AuthorizationException`, used for groupId authorization in KafkaConsumer is no longer used in the AdminClient API. The only one use case in AdminClient is for the topic based authorization in `createPartitions`.
   - For backward compatibility: The comment in Admin(https://github.com/apache/kafka/blob/2.6/clients/src/main/java/org/apache/kafka/clients/admin/Admin.java#L49-L51) is consistent with the test result by Gabor in https://github.com/apache/spark/pull/29729#issuecomment-712798812
   
   Some small suggestions:
   
   - Should we add some guides of the alternative method for the original usage of group.id based authorization? Maybe link the Kafka doc is enough.
   - IMO, It would be great to link all the investigations/tests in the PR description.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org