[jira] [Commented] (SPARK-55300) Visualize the behavior of startingOffset/endingOffset option in Kafka connector

["Kavpreet Grewal (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-55300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18055574#comment-18055574 ] 

Kavpreet Grewal commented on SPARK-55300:
-----------------------------------------

PR: https://github.com/apache/spark/pull/54080

> Visualize the behavior of startingOffset/endingOffset option in Kafka connector
> -------------------------------------------------------------------------------
>
>                 Key: SPARK-55300
>                 URL: https://issues.apache.org/jira/browse/SPARK-55300
>             Project: Spark
>          Issue Type: Improvement
>          Components: Documentation
>    Affects Versions: 4.2
>            Reporter: Kavpreet Grewal
>            Priority: Major
>              Labels: pull-request-available
>
> The guide doc of Kafka connector describes the behavior of startingTimestamp/endingTimestamp and related timestamp offset options as follows:
> [Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) - Spark 4.1.0 Documentation|https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html]
> {quote}The start point of timestamp when a query is started, a string specifying a starting timestamp for all partitions in topics being subscribed. Please refer the details on timestamp offset options below. If Kafka doesn't return the matched offset, the behavior will follow to the value of the option {{startingOffsetsByTimestampStrategy}}
> {quote}
> which can be illustrated in (ASCII style).
> We got some feedback from users that the text explanation in the guide doc may not be sufficient to come up with an illustration in their mind. Their feedback is something like “A picture is worth a thousand words”. So it would be great to visualize this and update the guide doc.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org