You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Shixiong Zhu (JIRA)" <ji...@apache.org> on 2018/10/01 18:13:00 UTC
[jira] [Resolved] (SPARK-25315) setting "auto.offset.reset" to
"earliest" has no effect in Structured Streaming with Spark 2.3.1 and Kafka
1.0
[ https://issues.apache.org/jira/browse/SPARK-25315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu resolved SPARK-25315.
----------------------------------
Resolution: Not A Bug
> setting "auto.offset.reset" to "earliest" has no effect in Structured Streaming with Spark 2.3.1 and Kafka 1.0
> --------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-25315
> URL: https://issues.apache.org/jira/browse/SPARK-25315
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.3.1
> Environment: Standalone; running in IDEA
> Reporter: Zhenhao Li
> Priority: Major
>
> The following code won't read from the beginning of the topic
> ```
> {code:java}
> val kafkaOptions = Map[String, String](
> "kafka.bootstrap.servers" -> KAFKA_BOOTSTRAP_SERVERS,
> "subscribe" -> TOPIC,
> "group.id" -> GROUP_ID,
> "auto.offset.reset" -> "earliest"
> )
> val myStream = sparkSession
> .readStream
> .format("kafka")
> .options(kafkaOptions)
> .load()
> .selectExpr("CAST(key AS STRING)", "CAST(value AS STRING)")
> myStream
> .writeStream
> .format("console")
> .start()
> .awaitTermination()
> {code}
> ```
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org