You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Jose Torres (JIRA)" <ji...@apache.org> on 2017/09/12 21:11:00 UTC

[jira] [Created] (SPARK-21988) Add default stats to StreamingExecutionRelation

Jose Torres created SPARK-21988:
-----------------------------------

             Summary: Add default stats to StreamingExecutionRelation
                 Key: SPARK-21988
                 URL: https://issues.apache.org/jira/browse/SPARK-21988
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 2.2.0
            Reporter: Jose Torres


StreamingExecutionRelation currently doesn't implement stats.

This makes some sense, but unfortunately the LeafNode contract requires that nodes which survive analysis implement stats, and StreamingExecutionRelation can indeed survive analysis when running explain() on a streaming dataframe.

This value won't ever be used during execution, because StreamingExecutionRelation does *not* survive analysis on the execution path; it's replaced with each batch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org