You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2016/11/10 13:29:58 UTC
[jira] [Updated] (SPARK-18342) HDFSBackedStateStore can fail to
rename files causing snapshotting and recovery to fail
[ https://issues.apache.org/jira/browse/SPARK-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-18342:
------------------------------
Assignee: Burak Yavuz
> HDFSBackedStateStore can fail to rename files causing snapshotting and recovery to fail
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-18342
> URL: https://issues.apache.org/jira/browse/SPARK-18342
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.0.1
> Reporter: Burak Yavuz
> Assignee: Burak Yavuz
> Priority: Critical
> Fix For: 2.0.2, 2.1.0
>
>
> The HDFSBackedStateStore renames temporary files to delta files as it commits new versions. It however doesn't check whether the rename succeeded. If the rename fails, then recovery will not be possible. It should fail during the rename stage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org