You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (JIRA)" <ji...@apache.org> on 2019/06/17 07:04:00 UTC
[jira] [Created] (SPARK-28074) [SS] Document caveats on using
multiple stateful operations in single query
Jungtaek Lim created SPARK-28074:
------------------------------------
Summary: [SS] Document caveats on using multiple stateful operations in single query
Key: SPARK-28074
URL: https://issues.apache.org/jira/browse/SPARK-28074
Project: Spark
Issue Type: Documentation
Components: Structured Streaming
Affects Versions: 3.0.0
Reporter: Jungtaek Lim
Please refer [https://lists.apache.org/thread.html/cc6489a19316e7382661d305fabd8c21915e5faf6a928b4869ac2b4a@%3Cdev.spark.apache.org%3E] to see rationalization of the issue.
tl;dr. Using multiple stateful operators without fully understanding how global watermark affects the query brings correctness issue.
While we may not want to fully block the operations given some users could leverage it with understanding of global watermark, it would be necessary for end users to warn on such breaking-correctness possibility, and provide alternative(s).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org