You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2022/11/06 22:40:00 UTC

[jira] [Assigned] (SPARK-41025) Introduce ComparableOffset as mixed-in interface for streaming offset

     [ https://issues.apache.org/jira/browse/SPARK-41025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-41025:
------------------------------------

    Assignee:     (was: Apache Spark)

> Introduce ComparableOffset as mixed-in interface for streaming offset
> ---------------------------------------------------------------------
>
>                 Key: SPARK-41025
>                 URL: https://issues.apache.org/jira/browse/SPARK-41025
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 3.4.0
>            Reporter: Jungtaek Lim
>            Priority: Major
>
> Currently, Spark does not care about the offset range when planning microbatch, and it has been data source implementation's responsibility to check the boundary of offset range.
> While not all types of offsets could provide comparison between two instances (some types may only be able to provide equality comparison), majority of types of offsets can do, which Spark can leverage it to perform the validation of offset range generally.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org