You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Anish Shrigondekar (Jira)" <ji...@apache.org> on 2025/10/29 20:59:00 UTC
[jira] [Resolved] (SPARK-54072) [State store integrity] Make sure we don't upload empty files in RocksDB snapshot
[ https://issues.apache.org/jira/browse/SPARK-54072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anish Shrigondekar resolved SPARK-54072.
----------------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
Issue resolved by pull request 52774
[https://github.com/apache/spark/pull/52774]
> [State store integrity] Make sure we don't upload empty files in RocksDB snapshot
> ---------------------------------------------------------------------------------
>
> Key: SPARK-54072
> URL: https://issues.apache.org/jira/browse/SPARK-54072
> Project: Spark
> Issue Type: Task
> Components: Structured Streaming
> Affects Versions: 4.1.0
> Reporter: B. Micheal Okutubo
> Assignee: B. Micheal Okutubo
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> This change is to make sure the RocksDB snapshot zip file doesn't contain empty files (except RocksDB log file). Other files such as OPTION, manifest, metadata are not expected to be empty in any situation.
> Introduced a conf {{verifyNonEmptyFilesInZip}} to make sure we can turn this off if needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org