[jira] [Created] (SPARK-30774) The default checkpointing interval is not as claimed in the comment.

["Kyle Krueger (Jira)" <ji...@apache.org>]

Kyle Krueger created SPARK-30774:
------------------------------------

             Summary: The default checkpointing interval is not as claimed in the comment.
                 Key: SPARK-30774
                 URL: https://issues.apache.org/jira/browse/SPARK-30774
             Project: Spark
          Issue Type: Bug
          Components: DStreams
    Affects Versions: 2.4.5
            Reporter: Kyle Krueger


[https://github.com/apache/spark/blob/71737861531180bbda9aec8d241b1428fe91cab2/streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala#L199-L203MajorMajor]

The checkpoint duration is set to be the window duration, maybe the idea in the old comment wanting to set to the higher of 10s or window-size is no longer relevant.

I propose we either adapt the comment to just say to just say that we set the checkpoint duration to the window size and clean up how that value is set, or we change the code to do as the comment remarks.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org