You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2020/06/15 14:50:00 UTC
[jira] [Resolved] (SPARK-31990) Streaming's state store
compatibility is broken
[ https://issues.apache.org/jira/browse/SPARK-31990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-31990.
-----------------------------------
Fix Version/s: 3.1.0
3.0.1
Resolution: Fixed
Issue resolved by pull request 28830
[https://github.com/apache/spark/pull/28830]
> Streaming's state store compatibility is broken
> -----------------------------------------------
>
> Key: SPARK-31990
> URL: https://issues.apache.org/jira/browse/SPARK-31990
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 3.0.0
> Reporter: Xiao Li
> Assignee: Takeshi Yamamuro
> Priority: Blocker
> Labels: correctness
> Fix For: 3.0.1, 3.1.0
>
>
> [This line|https://github.com/apache/spark/pull/28062/files#diff-7a46f10c3cedbf013cf255564d9483cdR2458] of [https://github.com/apache/spark/pull/28062] changed the order of groupCols in dropDuplicates(). Thus, the executor JVM could probably crash, throw a random exception or even return a wrong answer when using the checkpoint written by the previous version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org