You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by as...@apache.org on 2025/10/31 17:12:31 UTC
(spark) branch master updated (610bb52242c7 -> 6d5f111cc86a)

This is an automated email from the ASF dual-hosted git repository.

ashrigondekar pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


    from 610bb52242c7 [SPARK-54104][GEO][SQL] Disallow casting geospatial types to/from other data types
     add 6d5f111cc86a [SPARK-51972][SS] Introduce State Store file integrity verification using checksum

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/spark/internal/LogKeys.java    |   1 +
 .../src/main/resources/error/error-conditions.json |   7 +
 .../spark/sql/errors/QueryExecutionErrors.scala    |  17 +
 .../org/apache/spark/sql/internal/SQLConf.scala    |  11 +
 .../checkpointing/CheckpointFileManager.scala      |   2 +
 .../ChecksumCheckpointFileManager.scala            | 523 +++++++++++++++++++++
 .../state/HDFSBackedStateStoreProvider.scala       |  54 ++-
 .../sql/execution/streaming/state/RocksDB.scala    |  13 +-
 .../streaming/state/RocksDBFileManager.scala       |  54 ++-
 .../sql/execution/streaming/state/StateStore.scala |  59 +++
 .../streaming/state/StateStoreChangelog.scala      |  11 +-
 .../execution/streaming/state/StateStoreConf.scala |   3 +
 .../StateDataSourceTransformWithStateSuite.scala   |   3 +-
 .../streaming/CheckpointFileManagerSuite.scala     |   5 +-
 .../ChecksumCheckpointFileManagerSuite.scala       | 213 +++++++++
 .../FailureInjectionCheckpointFileManager.scala    |   4 +-
 .../RocksDBCheckpointFailureInjectionSuite.scala   |  11 +-
 .../streaming/state/RocksDBStateStoreSuite.scala   |  11 +
 .../execution/streaming/state/RocksDBSuite.scala   |   4 +
 .../streaming/state/StateStoreSuite.scala          | 292 +++++++++++-
 20 files changed, 1272 insertions(+), 26 deletions(-)
 create mode 100644 sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/checkpointing/ChecksumCheckpointFileManager.scala
 create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ChecksumCheckpointFileManagerSuite.scala


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org