You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Snoot.io (Jira)" <ji...@apache.org> on 2023/04/13 03:49:00 UTC
[jira] [Commented] (SPARK-43118) Remove unnecessary assert for UninterruptibleThread in KafkaMicroBatchStream
[ https://issues.apache.org/jira/browse/SPARK-43118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711668#comment-17711668 ]
Snoot.io commented on SPARK-43118:
----------------------------------
User 'jerrypeng' has created a pull request for this issue:
https://github.com/apache/spark/pull/40767
> Remove unnecessary assert for UninterruptibleThread in KafkaMicroBatchStream
> ----------------------------------------------------------------------------
>
> Key: SPARK-43118
> URL: https://issues.apache.org/jira/browse/SPARK-43118
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 3.3.2
> Reporter: Boyang Jerry Peng
> Priority: Minor
>
> The assert
>
> {code:java}
> assert(Thread.currentThread().isInstanceOf[UninterruptibleThread]) {code}
>
> found [https://github.com/apache/spark/blob/master/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchStream.scala#L239]
>
> is not needed. The reason is the following
>
> # This assert was put there due to some issues when the old and deprecated KafkaOffsetReaderConsumer is used. The default offset reader implementation has been changed to KafkaOffsetReaderAdmin which no longer require it run via UninterruptedThread.
> # Even if the deprecated KafkaOffsetReaderConsumer is used, there are already asserts in that impl to check if it is running via UninterruptedThread e.g. [https://github.com/apache/spark/blob/master/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReaderConsumer.scala#L130] thus the assert in KafkaMicroBatchStream is redundant.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org