You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dylan Wong (Jira)" <ji...@apache.org> on 2025/08/15 04:57:00 UTC

[jira] [Created] (SPARK-53288) Fix assertion error when using StreamingGlobalLimit and Checkpoint v2

Dylan Wong created SPARK-53288:
----------------------------------

             Summary: Fix assertion error when using StreamingGlobalLimit and Checkpoint v2
                 Key: SPARK-53288
                 URL: https://issues.apache.org/jira/browse/SPARK-53288
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 4.1.0
            Reporter: Dylan Wong


Checkpoint V2 has an assertion:
 {{assert(      ret.length == getStateInfo.numPartitions,
      s"ChekpointInfo length: ${ret.length}, numPartitions: ${getStateInfo.numPartitions}")}}
But StreamingGlobalLimit has only one partition.

Fix should be just relaxing this check for operators that have only 1 partition.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org