You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Shixiong Zhu (JIRA)" <ji...@apache.org> on 2018/08/06 22:33:00 UTC
[jira] [Resolved] (SPARK-24161) Enable debug package feature on
structured streaming
[ https://issues.apache.org/jira/browse/SPARK-24161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu resolved SPARK-24161.
----------------------------------
Resolution: Fixed
Assignee: Jungtaek Lim
Fix Version/s: 2.4.0
> Enable debug package feature on structured streaming
> ----------------------------------------------------
>
> Key: SPARK-24161
> URL: https://issues.apache.org/jira/browse/SPARK-24161
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 2.3.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
> Priority: Major
> Fix For: 2.4.0
>
>
> Currently, debug package has a implicit class which matches Dataset to provide debug features on Dataset class. It doesn't work with structured streaming: it requires query is already started, and the information can be retrieved from StreamingQuery, not Dataset. For the same reason, "explain" had to be placed to StreamingQuery whereas it exists on Dataset.
> This issue tracks effort to enable debug package feature on structured streaming. Unlike batch, it may have some restrictions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org