You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2023/04/13 08:11:00 UTC
[jira] [Resolved] (SPARK-43118) Remove unnecessary assert for UninterruptibleThread in KafkaMicroBatchStream
[ https://issues.apache.org/jira/browse/SPARK-43118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jungtaek Lim resolved SPARK-43118.
----------------------------------
Fix Version/s: 3.5.0
Resolution: Fixed
Issue resolved by pull request 40767
[https://github.com/apache/spark/pull/40767]
> Remove unnecessary assert for UninterruptibleThread in KafkaMicroBatchStream
> ----------------------------------------------------------------------------
>
> Key: SPARK-43118
> URL: https://issues.apache.org/jira/browse/SPARK-43118
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 3.3.2
> Reporter: Boyang Jerry Peng
> Assignee: Boyang Jerry Peng
> Priority: Minor
> Fix For: 3.5.0
>
>
> The assert
>
> {code:java}
> assert(Thread.currentThread().isInstanceOf[UninterruptibleThread]) {code}
>
> found [https://github.com/apache/spark/blob/master/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchStream.scala#L239]
>
> is not needed. The reason is the following
>
> # This assert was put there due to some issues when the old and deprecated KafkaOffsetReaderConsumer is used. The default offset reader implementation has been changed to KafkaOffsetReaderAdmin which no longer require it run via UninterruptedThread.
> # Even if the deprecated KafkaOffsetReaderConsumer is used, there are already asserts in that impl to check if it is running via UninterruptedThread e.g. [https://github.com/apache/spark/blob/master/connector/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReaderConsumer.scala#L130] thus the assert in KafkaMicroBatchStream is redundant.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org