[GitHub] [spark] HeartSaVioR commented on a change in pull request #32928: [WIP][SPARK-35784] Implementation for RocksDB instance

[GitBox <gi...@apache.org>]

HeartSaVioR commented on a change in pull request #32928:
URL: https://github.com/apache/spark/pull/32928#discussion_r655964610



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBFileManager.scala
##########
@@ -153,6 +156,49 @@ class RocksDBFileManager(
     logInfo(s"Saved checkpoint file for version $version")
   }
 
+  /**
+   * Load all necessary files for specific checkpoint version from DFS to given local directory.
+   * If version is 0, then it will deleted all files in the directory. For other versions, it
+   * ensures that only the exact files generated during checkpointing will be present in the
+   * local directory.
+   */
+  def loadCheckpointFromDfs(version: Long, localDir: File): RocksDBCheckpointMetadata = {

Review comment:
       If I understand correctly, commit 80154b2 should be the only change we need to review here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org