You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Genmao Yu (JIRA)" <ji...@apache.org> on 2019/04/09 10:40:01 UTC

[jira] [Created] (SPARK-27413) Keep the same epoch pace between driver and executor.

Genmao Yu created SPARK-27413:
---------------------------------

             Summary: Keep the same epoch pace between driver and executor.
                 Key: SPARK-27413
                 URL: https://issues.apache.org/jira/browse/SPARK-27413
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 3.0.0
            Reporter: Genmao Yu


The pace of epoch generation in driver and epoch pulling in executor is different. It will result in many empty epochs for partition if the epoch pulling interval is larger than epoch generation. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org